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Executive  summary 

Changes  in  worldwide  strategic  relationships  since  9/11  and  lessons 
learned  from  Operations  Iraqi  Freedom,  Enduring  Freedom,  and 
New  Dawn  demonstrate  the  importance  of  Navy  personnel  being  able 
to  communicate  in  foreign  languages  and  to  comprehend  foreign 
cultures.  The  Center  for  Information  Dominance’s  Center  for  Lan¬ 
guage,  Regional  Expertise,  and  Culture  (CLREC)  plays  a  central  role 
in  training  Department  of  the  Navy  personnel  in  the  language  skills 
and  cultural  knowledge  required  to  effectively  carry  out  their  mis¬ 
sions.  CLREC  provides  predeployment  cultural  orientation  training 
in  a  variety  of  forms,  such  as  (1)  group  instruction  facilitated  by  Mo¬ 
bile  Training  Teams,  (2)  semiautomated  group  presentations  facili¬ 
tated  by  a  member  of  the  command  being  trained,  (3)  individual 
instruction  using  automated  and  nonautomated  presentations,  and 
(4)  one-on-one  language  tutoring  for  flag  officers  who  will  soon  take 
an  assignment  in  which  linguistic  training  would  be  beneficial. 

For  the  following  reasons,  CLREC  wants  to  improve  its  current  pro¬ 
cesses  and  procedures  for  soliciting  customer  feedback:  to  determine 
how  well  its  products  are  being  accepted,  to  characterize  the  effects 
of  its  work,  and  to  identify  whether  there  are  unmet  needs  that 
should  be  addressed.  CLREC  turned  to  CNA  to  provide  feasible  op¬ 
tions  for  providing  feedback  from  customers  after  mission  perfor¬ 
mance,  instead  of  immediately  after  predeployment  training  (the 
current  process) .  CLREC  wants  these  options  designed  to  provide  in¬ 
formation  about  (a)  whether  training  was  applied  during  deploy¬ 
ments  and  (b)  whether  the  skills  that  were  applied  had  a  beneficial 
effect  on  mission  accomplishment  and  minimizing  liberty  incidents. 

To  develop  options  for  CLREC,  we  took  a  four-part  approach: 

1 .  Investigate  how  CLREC  and  other  organizations  have  assessed 
their  I  .RFC  training. 

2.  Develop  draft  survey  questions  for  a  Navy  CLREC  assessment. 
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3.  Investigate  how  CLREC  and  other  organizations  have  assessed 
their  I  RFC  ,  training. 

4.  Get  feedback  on  our  draft  survey  questions  from  a  variety  of 
knowledgeable  individuals,  including  sailors  serving  on  the 
2011  Southern  Partnership  Station  deployment. 

5.  Recommend  methods  for  deploying  the  Navy’s  CLREC  as¬ 
sessment  at  appropriate  times  and  places. 

From  our  analyses,  we  found  the  following: 

•  Assessing  whether  training  was  effective  in  supporting  mission 
objectives  needs  to  be  done  at  a  later  time,  not  immediately  af¬ 
ter  training  has  occurred. 

•  Data  and  analyses  should  focus  on  determining  needs  for  train¬ 
ing  rather  than  on  providing  tightly  defined  input-output  or  re¬ 
turn-on-investment  models  of  dollars  for  training  and  dollars  of 
benefit.  A  better  goal  would  be  to  provide  return-on- 
expectations  evidence  showing  that  sailors  were  prepared  and 
responded  well  to  situations  that  required  LREC  skills. 

•  Because  of  the  high  cost  of  developing  tests  of  proficiency,  self- 
assessments  will  probably  need  to  be  used  as  a  principal  indica¬ 
tor  for  assessing  the  results  of  training. 

•  Compared  with  the  other  services,  the  Navy  has  provided  train¬ 
ing  to  a  lower  percentage  of  personnel,  but  the  Navy  compares 
favorably  in  terms  of  such  important  indicators  as  sailors’  satis¬ 
faction  with  LREC  training  and  its  beneficial  results. 

•  Ideally,  an  assessment  of  Navy  CLREC  training  would  include 
collecting  survey/interview  data  from  sailors  and  their  offic¬ 
ers/unit  commanders.  We  developed  a  series  of  survey  instru¬ 
ments  and  interview  guides  that  are  tailored  to  assess  the  major 
CLREC  products.  For  example,  flag  officers  would  be  request¬ 
ed  to  complete  a  short  five-item  survey  or  interview. 

Various  organizations  could  field  a  survey  for  CLREC  and  collect  data 
(e.g.,  Defense  Manpower  Data  Center;  Navy  Personnel  Research, 
Studies,  and  Technology;  federally  funded  research  and  development 
centers;  and  survey  research  organizations).  CLREC  could  also  ex¬ 
plore  with  the  Navy  Education  and  Training  Command  whether  se¬ 
lected  instruments  can  be  embedded  in  Navy  Knowledge  Online. 


Chapter  1:  Introduction 

Changes  in  worldwide  strategic  relationships  since  9/11  and  lessons 
learned  from  Operations  Iraqi  Freedom,  Enduring  Freedom,  and 
New  Dawn  demonstrate  how  important  it  is  for  Navy  personnel  to 
have  skills  in  foreign  languages  and  understanding  of  foreign  cul¬ 
tures  [1].  The  Cooperative  Strategy  for  21st  Century  Seapower  (CS- 
21)  emphasizes  the  need  to  develop  relationships  with  emerging 
partners  to  develop  Global  Maritime  Partnerships  (GMP)  and  foster 
Theater  Security  Cooperation  (TSC).  GMP  and  TSC  improve  our 
Navy’s  ability  to  secure  sea  lanes  for  international  trade  and  to  police 
the  littorals  to  improve  the  safety  of  coastal  populations  and  stability 
of  coastal  regions.  Working  more  closely  with  international  partners 
in  day-to-day  operations  also  improves  the  Navy’s  ability  to  render 
humanitarian  assistance  and  disaster  relief  when  required.  By  know¬ 
ing  more  about  the  values,  beliefs,  behaviors,  and  norms  of  foreign 
cultures,  Department  of  the  Navy  (DON)  personnel  experience  less 
culture  shock  and  can  communicate  with  foreign  peoples  more  effi¬ 
ciently.  This  makes  them  both  better  ambassadors  of  our  Navy  and 
our  nation  and  a  more  mission-capable  force.1 

The  Center  for  Information  Dominance’s  Center  for  Language,  Re¬ 
gional  Expertise,  and  Culture  (CLREC)  plays  a  central  role  in  train¬ 
ing  DON  personnel  in  the  language  skills  and  cultural  knowledge 
required  to  effectively  carry  out  their  missions.  CLREC  provides  pre¬ 
deployment  cultural  orientation  training  to  sailors  in  a  variety  of 
forms  that  include  but  are  not  limited  to  the  following: 

1.  Group  instruction  facilitated  by  Mobile  Training  Teams 
(MTTs) 

2.  Semiautomated  group  presentations  facilitated  by  a  member 
of  the  command  being  trained 


1.  The  need  for  Navy  language  and  culture  training  will  not  go  away  when 
U.S.  forces  leave  Iraq  and  Afghanistan;  this  is  a  basic  change  in  defense 
strategy.  See,  for  example,  the  2010  Quadrennial  Defense  Review  Report  [2]. 
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3.  Individual  instrviction  via 


a.  Paced  Navy  e-Learning  (NeL) 

b.  Automated  and  nonautomated  presentations 

In  addition,  CLREC  provides  sailors  predeployment  foreign  language 
familiarization  training  via  MTTs  and  through  paced  training  materi¬ 
als  and  products  developed  by  the  government  or  commercial  indus¬ 
try.  It  also  coordinates  language  training  for  flag  officers  who  will 
soon  take  an  assignment  that  would  benefit  from  linguistic  training. 

The  goal  of  this  language,  regional  expertise,  and  cultural  (LREC) 
training  is  to  reduce  culture  shock  and  develop  rudimentary  lan¬ 
guage  skills  among  sailors.  It  is  believed  that  sailors  who  understand 
the  culture  and  can  communicate  even  a  little  are  better  able  to  an¬ 
ticipate  the  attitudes  and  actions  of  those  they  will  meet  and  those 
with  whom  they  will  work  while  overseas. 


Issues 


CLREC  desires  to  improve  its  current  processes  and  procedures  for 
soliciting  customer  feedback.  CLREC  is  seeking  to: 

•  determine  how  well  its  products  are  being  accepted; 

•  characterize  the  effects  of  its  work  on  individual  and  unit  read¬ 
iness  and  fleet  operations;  and 

•  identify  whether  there  are  unmet  needs  that  should  be  ad¬ 
dressed. 

Figure  1  graphically  depicts  the  goals  that  are  attained  by  CLREC’s 
current  assessment  process  and  what  it  wants  to  add  to  that  process. 

The  current  feedback  process  allows  CLREC  to  gather  students’  im¬ 
pressions  immediately  after  their  training,  as  shown  by  the  solid  dark 
blue  arrows.  The  intent  of  our  study  is  to  assist  CLREC  in  its  efforts  to 
provide  feedback  loops  after  mission  performance  (shown  by  the 
dashed  red  lines).  We  do  this  by  developing  survey  questions  for  a 
Navy  CLREC  assessment  and  by  recommending  methods  for  deploy¬ 
ing  the  assessment. 
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Figure  1 .  Current  and  proposed  feedback  loops  for  Navy's  CLREC 
training  assessments 
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There  are  two  parts  of  the  proposed  feedback  loops.  One  is  to  assess 
the  impact  of  the  CLREC  program  on  the  behavior  and  performance 
of  Navy  personnel  during  their  mission,  which  we  have  labeled  as 
“Impact.”  The  other  part  is  to  assess  needs  for  training,  as  communi¬ 
cated  by  sailors  who  are  finishing,  or  just  recently  finished,  their  mis¬ 
sion  assessment  (“Needs”). 

One  way  to  conceptualize  the  impact  portion  of  the  new  feedback  av¬ 
enues  is  by  using  Kirkpatrick’s  levels  of  training  evaluation  [3,  4], 
shown  in  table  1 .  The  four  levels  can  be  succinctly  described  as  reac¬ 
tion,  learning,  behavior,  and  results.  CLREC  currently  gets  systematic 
feedback  immediately  after  training,  measuring  the  students’  reac¬ 
tions  to  the  training,  how  satisfied  they  are  with  the  training,  and 
what  they  liked/ disliked  about  the  class  (Kirkpatrick  Level  1) .  Level  2 
assessments,  in  contrast,  include  evaluation  of  whether  participants 
acquire  the  knowledge,  skills,  and  attitudes  based  on  their  participa¬ 
tion  in  CLREC  training. 

CLREC  wants  to  add  evaluations  primarily  at  Kir  kpatrick  Levels  2,  3, 
and  4  to  its  current  evaluation  processes.  Level  3  evaluations  involve 
assessing  how  much  participants  apply  what  they  learned  in  CLREC 
training  when  they  get  to  their  deployment  situations.  Level  4  would 
involve  assessing  whether  desired  outcomes  occur  as  a  result  of 
CLREC  training. 
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Table  1 

.  Kirkpatrick's  four  levels  of  training  evaluation  (adapted  from  [3] 
and  [4]) 

Level 

Description  of  the  evaluation  level 

1 

Degree  to  which  participants  react  favorably  to  the  training  event 

2 

Degree  to  which  participants  acquire  the  intended  knowledge, 
skills,  and  attitudes  based  on  their  participation  in  the  learning 
event. 

3 

Degree  to  which  participants  apply  what  they  learned  during  train¬ 
ing  when  they  get  back  to  the  job. 

4 

Degree  to  which  targeted  outcomes  occur,  as  a  result  of  the  learn¬ 
ing  events  and  subsequent  reinforcement;  includes  return  on  in¬ 
vestment  and  return  on  expectation. 

Some  training  evaluators  [5]  describe  a  fifth  level,  return  on  invest¬ 
ment  (monetary  return  minus  costs).  The  Kirkpatricks  themselves  see 
return  on  investment  as  part  of  the  fourth  level,  but  they  prefer  to 
advocate  “return  on  expectation”  [6].  Unlike  return  on  investment, 
the  Kirkpatricks’  return  on  expectation  involves  both  qualitative  and 
quantitative  measurement  of  all  benefits  realized  as  a  result  of  train¬ 
ing — not  only  those  that  can  be  measured  quantitatively  or  converted 
into  monetary  values,  as  would  be  the  case  with  return  on 
investment. 

There  are  several  challenges  to  providing  feedback  loops,  as  shown  in 
figure  1  and  table  1.  First  is  the  need  to  identify  sailors  who  use 
CLREC  services  in  a  way  that  would  allow  them  to  be  contacted  dur¬ 
ing — or,  more  likely,  after — their  deployments.  Second,  sailors  might 
not  remember  very  much  about  their  predeployment  training  since 
the  time  span  between  receiving  training  and  returning  from  de¬ 
ployment  can  be  six  or  more  months.  Third,  sailors  have  had  enough 
time  on  deployment  to  have  learned  skills  since  their  training,  and  it 
will  be  difficult  for  sailors  to  distinguish  between  skills  that  they 
learned  before  deployment  and  those  learned  during  deployment. 

To  help  CLREC  meet  these  challenges,  we  took  a  four-part  approach: 
(1)  develop  draft  survey  questions  for  a  Navy  CLREC  assessment,  (2) 
investigate  how  CLREC  and  other  organizations  have  assessed  their 
LREC  training,  (3)  get  feedback  on  our  draft  survey  questions  from  a 
variety  of  knowledgeable  people,  including  sailors  serving  on  the 
2011  Southern  Partnership  Station  deployment,  and  (4)  recommend 
methods  for  deploying  the  Navy’s  CLREC  assessment  at  appropriate 


times  and  places.  The  second  step  involved  visiting  CLREC  and  oth¬ 
er  commands  and  offices  of  the  armed  forces  that  train  and  survey 
servicemembers  in  language  or  culture  to  derive  “lessons  learned”  in 
terms  of  what  questions  to  ask,  when  to  ask  them,  and  how  to  ask 
them. 

Organization  of  this  report 

This  report  is  organized  in  four  chapters: 

•  Chapter  1,  Introduction 

•  Chapter  2,  Previous  assessments  of  language,  regional  expertise,  and 
culture,  reviews  the  reasons  for  assessing  LREC  training  and 
findings  from  previous  studies  and  activities  related  to  LREC 
training  assessment — both  within  and  outside  the  DON. 

•  Chapter  3,  LREC  questions,  explains  and  reviews  the  survey  ques¬ 
tions  that  we  developed  and  then  modified  on  the  basis  of 
feedback. 

•  Chapter  4,  Alternatives  and  recommendations,  describes  different 
ways  that  CLREC  can  field  these  surveys  in  an  efficient  and 
economical  way. 
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Chapter  2:  Previous  assessments  of  language, 
regional  expertise,  and  culture 

As  part  of  our  study  of  how  CLREC  can  best  assess  the  training  and 
information  that  it  provides  to  the  Navy,  we  reviewed  information 
from  a  number  of  different  sources,  including  policy  documents,  re¬ 
search  reports,  and  survey  results.  We  also  interviewed  people 
throughout  the  government  who  have  familiarity  with  language  and 
culture  programs  and  assessment.  This  chapter  summarizes  the  re¬ 
sults  of  this  research. 


Congress 

Congress  has  been  concerned  with  the  skills  and  expertise  needed  by 
today’s  military  during  Operation  Iraqi  Freedom  (OIF)  and  Opera¬ 
tion  Enduring  Freedom  (OEF).  As  the  counterinsurgency  role  of 
troops  increased  in  these  operations,  key  committees  investigated 
whether  the  Department  of  Defense  (DOD)  and  the  services  were 
addressing  language,  culture,  and  regional  studies  in  the  general 
forces. 

The  House  Armed  Services  Committee  (HASC)  Oversight  and  Inves¬ 
tigations  (O&I)  Staff  has  produced  two  reports  addressing  LREC  is¬ 
sues.  The  first  [7],  Building  Language  Skills  and  Cultural  Competencies  in 
the  Military:  DOD’s  Challenge  in  Today’s  Educational  Environment,  identi¬ 
fied  for  further  study  the  question  of  how  well  the  services  provided 
language  and  cultural  awareness  training  for  deploying  units.  They 
found  that  the  services  needed  to  measure  language,  regional  exper¬ 
tise,  and  cultural  awareness,  and  develop  the  relationship  between 
these  measures  and  overall  unit  readiness.  In  2010,  the  HASC  O&I 
Subcommittee  issued  a  second  report  [8]  on  military  language  and 
cultural  competencies.  It  found  that  the  service  policies  still  fell  short 
of  recognizing  language,  regional  expertise,  and  cultural  awareness 
as  core  competencies  essential  to  mission  success. 

The  Government  Accountability  Office  (GAO)  also  has  investigated 
DOD’s  and  the  services’  work  in  developing  language  skills  and 
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regional  proficiency.  In  2009,  it  found  that  DOD  lacked  methods  to 
assess  regional  expertise  requirements  and  skills  [9].  When  GAO 
reexamined  language  and  culture  training  in  2011,  it  noted  that 
DOD  has  recognized  that  its  ability  to  identify  general  purpose  forces 
that  have  language  and  culture  knowledge  and  skills  will  be  critical  to 
managing  these  forces  in  the  future  [10].  The  services,  however,  were 
not  capturing  information  in  training  and  personnel  systems  on  the 
completion  of  language  and  culture  training  and  corresponding  pro¬ 
ficiency  gained  from  training.  The  services  do  not  have  the  infor¬ 
mation  they  need  to  effectively  leverage  the  language  and  culture 
knowledge  and  skills  of  these  forces  when  making  individual  assign¬ 
ments  and  assessing  future  operational  needs. 

Defense  Language  and  National  Security  Office  (DLNSEO) 

DLNSEO  has  been  involved  in  establishing  policies  and  coordinating 
among  services  with  respect  to  language,  regional  expertise,  and  cul¬ 
ture.  It  has  been  involved  with  tracking  Regional  Proficiency  (RP)  in 
the  force  but  has  not  yet  implemented  tests  for  culture  or  regional 
experience. 

DLNSEO  is  working  with  the  Center  for  Advanced  Study  of  Language 
(CASL)  to  develop  a  means  to  rate  the  regional  experiences  of  per¬ 
sonnel  according  to  the  RP  guidelines  in  DOD  Instruction  5160.70, 
Management  of  DoD’s  Language  and  Regional  Proficiency  Capabilities.  This 
rating  project  is  an  examination  of  experiential  data  that  are  or 
should  be  contained  in  service  databases  (primarily  personnel  data¬ 
bases)  .  DLNSEO  and  CASL  use  a  weighted  algorithm  to  derive  a  rat¬ 
ing  (not  a  score).  The  ratings  should  allow  them  to  swiftly  identify 
who  has  experience  in  any  of  the  regions. 

The  Defense  Manpower  Data  Center  conducted  a  DLNSEO- 
sponsored  survey  in  November  2011  on  assessing  regional  proficien¬ 
cy.  While  the  survey  results  have  not  yet  been  released,  the  survey  in¬ 
cludes  a  number  of  questions  on  education,  language  knowledge, 
cultural  and  family  background,  and  military  training.  The  questions 
on  deployment  history  and  predeployment  training  provide  exam¬ 
ples  of  questions  that  could  be  used  as  a  benchmark  with  which  to 
compare  other  survey  results. 
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Defense  Manpower  Data  Center  (DMDC) 

Over  the  last  several  years,  DMDC  has  performed  annual  Status  of 
Forces  Surveys  of  Active  Duty  Members  (SOFS-A)  and  Reserve  Com¬ 
ponent  Members  (SOFS-R)  that  include  items  to  identify  members 
who  have  deployed  in  the  past  24  months  and  who  are  currently 
deployed. 

The  June  2010  SOFS-A  and  January  2011  SOFS-R  included  the  items 
requested  by  DLNSEO  that  addressed  language  and  culture.  These 
items  considered: 

•  Whether  the  members  had  received  any  language,  cultural,  or 
regional  training  before  deployment 

•  How  satisfied  they  were  with  the  training 

•  Whether  the  Language,  Region,  and  Culture  (LRC)  training 
met  the  specified  objectives 

•  Whether  the  training  helped  them  in  performing  their  job 

DMDC’s  findings  [11]  have  been  reported  by  service/reserve  com¬ 
ponent.  Within  each  service/reserve  component,  data  are  further 
broken  out  by  enlisted  or  officer  and  by  rank.  Additional  breakouts — 
by  marital  status,  gender,  ethnicity,  and  broad  location  categories — 
are  also  provided  for  many  of  the  questions. 

The  survey  contains  many  of  the  elements  that  would  be  useful  for  a 
high-level  program  assessment,  including  the  Kirkpatrick  framework. 
For  example,  the  first  step  in  an  evaluation  would  be  to  find  out  if  any 
LRC  training  was  received.  The  survey  asks  whether  the  respondent 
has  had  any  LRC  training.  DMDC  found  that  64  percent  of  troops 
deploying  for  either  OEF  or  OIF  had  received  cultural  training,  53 
percent  had  received  regional  training,  and  29  percent  had  received 
language  training.  Figure  2  shows  the  percentage  of  each  service  that 
received  language,  regional,  or  cultural  training.  It  shows  that  the 
Navy  provided  LRC  training  to  a  smaller  percentage  of  its  forces  than 
did  the  other  three  services.  The  figure  also  shows  that,  for  all  four 
services,  cultural  training  is  provided  more  often  than  regional  train¬ 
ing;  language  training  is  the  least  commonly  provided  LRC  training. 
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The  DMDC  svirvey  also  provides  indicators  of  the  outcomes  of  train¬ 
ing.  Customer  satisfaction  is  considered  a  Kirkpatrick  Level  1  metric. 
It  is  also  frequently  used  as  an  assessment  measure  for  service  deliv¬ 
ery.  Figure  3  shows  the  percentage  of  deployed  members  who  were 
satisfied  with  their  LRC  training.  Among  the  four  services,  the  high¬ 
est  percentage  of  satisfied  LRC  participants  was  in  the  Navy  (46  per¬ 
cent),  followed  by  Marine  Corps  (40  percent),  Air  Force  (38 
percent),  and  Army  (35  percent).  The  Navy  was  significantly  higher 
than  the  other  services. 

Figures  2  and  3,  taken  together,  suggest  that  the  Navy  trains  a  smaller 
segment  of  its  deploying  population  in  LRC  skills;  however,  those  it 
trains  are  more  satisfied  with  the  training  they  receive. 

DMDC  also  asked  about  the  degree  to  which  LRC  training  was  bene¬ 
ficial.  First,  they  asked  whether  the  training  had  met  their  objectives, 
which  is  an  example  of  a  high-level  assessment  of  Kirkpatrick  level  2 
(see  figure  4) 

Figure  5  shows  that,  across  all  the  services,  officers  saw  greater  benefit 
to  their  LRC  training  than  did  enlisted  members.  It  also  shows  that 
Navy  and  Marine  Corps  officers  viewed  their  LRC  training  as  benefi¬ 
cial  more  often  than  any  other  group — with  67  percent  saying  their 
training  helped  them  perform  their  job  (Kirkpatrick  Levels  3  and  4) . 


Figure  2.  Percentage  of  active  duty  servicemembers  who  received 
LRC  training  before  deployment 
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Figure  3.  Percentage  of  deploying  active  duty  servicemembers  satisfied 
with  their  LRC  training  (Kirkpatrick  Level  1 ) 
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Figure  4.  Percentage  of  servicemembers  deployed  for  OEF  and/or  OIF 
saying  LRC  training  met  the  objectives  (Kirkpatrick  Level  2) 
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Figure  5. 
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In  contrast,  only  64  percent  of  Army  officers  and  56  percent  of  Air 
Force  officers  agreed  that  their  LRC  training  was  beneficial.  On  the 
enlisted  side,  the  highest  percentages  of  servicemembers  describing 
their  LRC  training  as  beneficial  were  Marines  (61  percent) ,  followed 
by  Navy  (59  percent) ,  Army  (55  percent)  and  Air  Force  (53  percent) . 

It  would  be  useful  for  the  survey  to  probe  into  additional  detail  on 
what  kinds  of  knowledge  the  respondents  received.  The  survey  asked 
respondents  whether  they  received  training  in  a  number  of  different 
areas,  such  as  religion,  geography,  social  organization,  and  political 
structure.  This  kind  of  feedback  is  useful  to  the  services’  LRC  trainers 
because  it  provides  additional  information  on  the  specific  areas  in 
which  respondents  received  training.  However,  additional  types  of 
feedback,  such  as  what  kinds  of  training  people  actually  used,  and  es¬ 
pecially  the  areas  where  they  would  desire  additional  training,  would 
be  a  useful  addition. 

Defense  Language  Institute  Foreign  Language  Center 

(DLIFLC) 


DLIFLC  is  the  world’s  largest  foreign  language  instructional  institu¬ 
tion.  In  addition  to  running  educational  programs  to  train 
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professional  linguists,  DLIFLC  supports  several  programs  that  pro¬ 
vide  training  to  general  purpose  forces  in  language  and  culture. 

DLIFLC  supports  selected  soldiers  in  the  general  purpose  force  with 
16  weeks  of  predeployment  training.  In  addition  to  education  in  re¬ 
gional  studies  and  culture,  the  language  training  is  designed  to  raise 
soldier  proficiency  from  0  to  1  on  the  interagency  Language 
Roundtable  (ILR)  scale  (basic  functionality) . 

The  assessment  of  general  purpose  soldiers  is  focused  on  when  they 
return  from  deployment.  Soldiers  usually  spend  a  week  at  Fort 
Campbell,  KY,  after  deployment.  During  this  week,  DLIFLC  surveys 
the  soldiers  to  learn  about  the  suitability  of  their  training  and  the 
kinds  of  language  and  culture  issues  that  they  were  involved  in  during 
their  deployment. 

A  second  major  language  and  culture  program  administered  by 
DLIFLC  is  the  Afghanistan/Pakistan  Hands  (AFPAK  Hands)  pro¬ 
gram.  The  AFPAK  Hands  Program  was  established  in  2009  to  create 
greater  continuity,  focus  and  persistent  engagement  on  Afghanistan 
and  Pakistan.  This  program  develops  a  cadre  of  military  and  civilian 
personnel  who  speak  the  local  language,  are  culturally  attuned,  and 
focus  on  regional  issues.  Participants  undergo  initial  training,  then 
deploy  into  assignments  in  the  region,  and  return  to  the  AFPAK  re¬ 
gion  after  undergoing  additional  training. 

DLIFLC  provides  the  language  and  culture  training  for  the  AFPAK 
Hands  program.  Participants  initially  receive  16  weeks  of  training,  in¬ 
cluding  language  training  that  is  designed  to  bring  them  up  to  the  1 
level  on  the  ILR  scale. 

DLIFLC  receives  feedback  from  AFPAK  Hands  members  at  various 
times.  Students  are  surveyed  after  completion  of  initial  training. 
Once  deployed  to  Afghanistan  or  Pakistan,  they  are  also  surveyed  in 
country  at  a  conference  where  all  trainees  are  brought  together  to 
learn  about  their  experiences  with  language  and  culture.  When 
participants  return  to  the  continental  United  States  (CONUS),  they 
are  assessed  again,  with  a  focus  on  a  needs  assessment  before  receiv¬ 
ing  additional  training.  During  debriefings  they  have  an  opportunity 
to  discuss  their  language  and  culture  experiences  and  training  needs 
in  a  group  session. 
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The  Defense  Equal  Opportunity  Management  Institute 
(DEOMI) 


The  Division  of  Cultural  Readiness  of  DEOMI  conducts  basic  and 
applied  cultural  research  to  inform  DOD  policy-makers;  it  develops, 
validates,  and  assesses  cultural  training;  and  it  provides  measurement, 
analytic  capability,  and  online  knowledge  management  tools  for  cul¬ 
ture  training  available  across  the  DOD  and  industry.  It  has  also  spon¬ 
sored  symposia  on  the  role  of  culture,  language,  and  regional 
expertise  in  military  operations. 

One  example  of  the  work  that  DEOMI  supports  is  the  paper,  “As¬ 
sessing  Organizational  Cross-Cultural  Competence  in  the  Military: 
Developing  a  Measure,”  by  Marinus  van  Driel  [12].  Dr.  van  Driel  ex¬ 
plains  how  DEOMI  developed,  pretested,  and  modified  a  measure  of 
military  servicemembers’  perceptions  of  their  units’  cross-cultural 
competence.  It  started  by  collecting  critical  incidents  mentioned  dur¬ 
ing  focus  groups  conducted  with  United  Special  Operations  Forces 
(SOF),  such  as  Army  Special  Forces  (Green  Berets)  or  Navy  Sea,  Air, 
Land  team  members  (SEALs) .  Critical  incidents  are  vignettes  that  are 
important  to  mission  accomplishment  and  also  illustrate  something 
about  the  competencies  needed  by  a  special  operator. 

The  critical  incidents  served  as  the  basis  for  59  items  reflecting  some 
aspects  of  the  performance,  preparation,  or  resources  related  to 
cross-cultural  competence.  DEOMI  researchers  included  these  items 
with  the  DEOMI  Equal  Opportunity  Climate  Survey  (DEOCS)  and 
asked  respondents  to  indicate  the  importance  of  cross-cultural  com¬ 
petence  attributes  to  their  unit  during  its  previous  deployment.  From 
this  pilot  test,  they  selected  17  items  that  respondents  agreed  were 
critically  important  to  cross-cultural  competence  for  special  opera¬ 
tions  forces.  These  17  items,  and  5  additional  items,  were  included 
for  additional  pilot  testing  as  part  of  the  DEOCS — obtained  from  474 
servicemembers  in  50  units  who  were  deployed  outside  CONUS 
(OCONUS).  Further  item  analysis  led  Dr.  van  Driel  to  recommend  a 
10-item  measure  of  units’  cross-cultural  competence  in  military  units. 
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Department  of  the  Navy 

Navy  Foreign  Language  Office 

The  Navy’s  Foreign  Language  Office  (N13F)  provides  guidance  to 
CT  .RFC  and  other  entities  within  the  Navy.  When  we  met  with  N13F 
staff,  they  said  that  they  wanted  CLREC  to  get  systematic  feedback, 
not  just  ad  hoc  feedback.  They  were  particularly  interested  in  having 
CLREC  get  information  on  how  much  their  products  and  services 
helped  sailors  (1)  perform  their  mission  better  or  (2)  keep  from  cre¬ 
ating  incidents  during  liberty.  N13F’s  philosophy  is  that  there  is  a  lim¬ 
ited  amount  of  time  to  train,  so  CT  .RFC  should  not  provide  any  more 
I  .RFC  training  than  necessary.  This  is  why  all  Navy  LREC  training  is 
“demand-based” — meaning  that  units  must  request  LREC  training  in 
order  to  receive  it.  In  other  words,  units  will  not  be  forced  to  receive 
LREC  training. 

On  one  hand,  N13F  notes  that  the  Navy’s  language,  regional  exper¬ 
tise,  and  culture  programs  need  to  be  different  from  those  in  other 
services  in  several  ways.  The  Navy  does  more  with  general  cultural 
competence  because  a  ship’s  cruise  can  take  a  sailor  from  Indonesia, 
to  India,  and  to  the  Persian  Gulf  within  a  six-month  period.  All  of 
these  would  be  port  calls — not  extensive  contact  with  foreign  peo¬ 
ples,  but  brief,  superficial  encounters. 

Most  of  the  Army’s  deploying  units,  on  the  other  hand,  can  rely  on 
the  fact  that  they  will  be  serving  around  Afghanistani  people,  often 
over  long  periods  of  time  where  it  makes  sense  to  learn  cross-cultural, 
regional,  and  language  competencies  in  more  depth. 

All  of  these  differences  between  the  Navy  and  Army  change  the  way 
CLREC  provides  Navy  LRC  training.  CLREC  provides  information  on 
a  larger  number  of  countries  because  Navy  personnel  are  likely  to  be 
deployed  anywhere  in  the  world.  This  is  why  Navy  personnel  can  re¬ 
quest  products  from  CLREC,  such  as  language  DVDs,  CDs,  lessons, 
and  operational  cultural  awareness  training  (OCAT). 

Some  of  the  Navy  commands  that  use  CLREC  products  and  services 
the  most  include  Seabees  (Construction  Battalions),  Military  Civil  Af¬ 
fairs  Units,  Explosive  Ordnance  Disposal  (EOD)  units,  and  the 
Expeditionary  Intelligence  Command.  There  is  also  some  demand 
from  Carrier  Strike  Groups. 
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The  client  needs  to  tell  CLREC  what  the  specific  mission  will  be  and 
work  with  CLREC’s  personnel  to  determine  the  appropriate  package 
of  language,  regional  expertise,  and  cultural  training  that  is  needed. 
The  package  for  a  unit  doing  reconstruction  might  be  different  from 
a  unit  doing  humanitarian  operations.  CLREC  also  provides  Navy 
personnel  with  training  on  how  to  work  with  interpreters,  if  that  is 
needed. 

Office  of  Naval  Research  (ONR) 

ONR  supports  a  variety  of  research  to  help  the  Navy  perform  its  mis¬ 
sions.  ONR  has  developed  a  program  called  ISLET,  which  stands  for 
Interservice  System  for  Language  Education  and  Training.  This  is  a 
way  to  learn  a  language  through  playing  games.  ISLET  currently  con¬ 
tains  training  in  five  different  languages.  CLREC  developed  a  tactical 
Swahili  module  for  them. 

Navy  Personnel  Research,  Studies,  and  Technology  (NPRST) 

NPRST,  based  in  Millington,  TN,  is  the  Navy’s  only  personnel-focused 
research  laboratory.  One  of  its  capabilities  is  to  provide  Navy-wide  sur¬ 
vey  development,  administration,  and  analysis.  It  operates  two  types  of 
surveys  within  this  mission.  One  is  a  scientific  survey  of  the  entire  Na¬ 
vy’s  Total  Force.  The  second  type  of  survey  is  a  “quick  poll,”  where 
NPRST  surveys  specific  populations  concerning  specific  topics,  such  as 
family  readiness,  knowledge  of  technology,  or  stress  management. 

CNA's  Center  for  Naval  Analyses 

CNA  is  a  nonprofit  research  organization  that  operates  the  Center 
for  Naval  Analyses  and  the  Institute  for  Public  Research.  The  Center 
for  Naval  Analyses  provides  research  and  analysis  services  to  the  mili¬ 
tary  to  help  improve  the  efficiency  and  effectiveness  of  our  national 
defense  efforts. 

In  earlier  research  related  to  LRC  training  [13],  Center  for  Naval 
Analyses’  Ed  Schmitz  and  his  colleagues  developed  metrics  for  the 
Navy’s  Bureau  of  Medicine  (BUMED)  to  assist  in  the  manning  and 
training  of  medical  personnel  for  performing  humanitarian  assis¬ 
tance  (HA)  and  stability,  security,  transition  and  reconstruction 
(SSTRO)  missions.  Our  focus  was  on  identifying  and  training  medical 
personnel  with  respect  to  the  language  and  cultural  aspects  of  their 
missions. 
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Figure  6  shows  a  diagram  of  the  model  that  Schmitz  and  his  col¬ 
leagues  developed.  On  the  left  side  are  the  inputs  to  the  delivery  of 
HA.  These  are  the  attributes  of  the  personnel  assigned,  characteris¬ 
tics  of  the  mission,  and  the  nature  of  additional  mission-specific  train¬ 
ing  that  is  provided. 


Figure  6.  A  model  of  evaluating  language  and  culture  training  [1 0] 


In  this  model,  the  personnel  assigned  to  the  HA  or  SSTRO  mission 
would  be  assessed  for  their  knowledge  of  culture  at  the  beginning  of 
the  mission.  This  would  typically  be  done  with  a  self-assessment 
against  the  previously  established  cultural  awareness  metrics.  Army 
Research  Institute  (ARI)  reports  such  as  Abbe,  Gulick,  and  Herman 
(2010)  [14]  or  Caligiuri  et  al.  (2011)  [15]  provide  examples  of  the 
kinds  of  assessment  tools  that  could  be  used.  If  any  additional  lan¬ 
guage  training  were  provided,  the  candidates  could  be  assessed,  ei¬ 
ther  through  the  Defense  Language  Proficiency  Test  (DLPT)  or 
through  a  self-assessment. 

In  this  model,  the  personnel  assigned  to  the  HA  or  SSTRO  mission 
would  be  assessed  for  their  knowledge  of  culture  at  the  beginning  of 
the  mission.  This  would  typically  be  done  with  a  self-assessment 
against  the  previously  established  cultural  awareness  metrics.  Army 
Research  Institute  (ARI)  reports  such  as  Abbe,  Gulick,  and  Herman 
(2010)  [14]  or  Caligiuri  et  al.  (2011)  [15]  provide  examples  of  the 
kinds  of  assessment  tools  that  could  be  used.  If  any  additional  lan¬ 
guage  training  were  provided,  the  candidates  could  be  assessed,  ei¬ 
ther  through  the  Defense  Language  Proficiency  Test  (DLPT)  or 
through  a  self-assessment. 


19 


Schmitz  and  his  coauthors  identified  two  types  of  measures  to  be  col¬ 
lected  at  the  service  delivery  point.  The  first  metric  was  the  identifica¬ 
tion  of  service  delivery  issues.  These  are  often  referred  to  as 
performance  indicators.  These  can  include  measures  of  the  frequen¬ 
cy  and  kinds  of  interactions  medical  personnel  had  with  patients  and 
host  nationals,  kinds  of  communication  problems  encountered,  and 
narratives  with  detailed  comments  on  issues  and  experiences. 

For  medical  personnel,  working  with  interpreters  was  an  important 
topic  that  needed  to  be  addressed.  Nearly  all  medical  personnel 
deployed  on  HA  or  SSTRO  missions  end  up  working  with  interpret¬ 
ers  at  some  point.  Previous  research  by  Hayes  and  Baker  [16]  had 
found  that  working  with  medical  interpreters  presented  specific  chal¬ 
lenges.  Thus,  the  assessments  of  service  delivery  needed  to  address 
the  person’s  experiences  in  working  with  interpreters.  This  feedback 
was  important  to  improving  training,  as  well  as  providing  guidance  to 
the  organization  providing  interpreter  services. 

A  second  category  of  measures  was  outcome  or  impact  indicators. 
These  measures  address  whether  the  performance  of  the  activity  was 
contributing  to  the  overall  program  goals.  Defining  the  outcome  of 
HA  or  SSTRO  missions  was  a  subject  of  considerable  discussion.  Past 
analysis  by  Nelson  et  al.  has  examined  such  metrics  as  the  number  of 
patients  treated  and  the  character  of  care  provided  [17].  Other  re¬ 
cent  studies,  such  as  Vernon’s  2008  report,  have  focused  on  attitudi- 
nal  measures  of  effectiveness  [18]. 

Schmitz  and  colleagues  developed  survey  instruments  to  assess  the 
training,  service  delivery,  and  mission  outcomes  in  two  cases:  the  de¬ 
ployment  of  Mercy  to  provide  humanitarian  assistance  in  2008  and 
medical  personnel  in  Afghanistan  performing  SSTRO  missions.  The 
model  was  most  fully  applied  in  Mercy’s  Pacific  mission.  Metrics  were 
collected  on  (1)  the  cultural  training  provided  for  medical  and  non¬ 
medical  personnel,  (2)  medical  personnel’s  assessment  of  service  de¬ 
livery  issues  and  medical  outcomes,  and  (3)  patient  surveys  providing 
their  assessment  of  delivery  problems  and  medical  outcomes. 

Previous  research  by  Flores  (2005)  had  found  that  indicators  of  pa¬ 
tient  feedback  on  medical  services  could  serve  as  a  reasonable  indica¬ 
tor  of  successful  treatment  [19].  Our  conceptual  model,  validating 


assessment  of  cultural  training  outcomes  to  mission  success,  remains 
a  promising  approach  for  future  exercises. 

Marine  Corps  Center  for  Advanced  Operational  Culture  Learning 
(CAOCL) 

CAOCL  began  in  2006  and  is  focused  on  support  for  general-purpose 
forces.  CAOCL  is  operationally  oriented,  concentrating  its  training 
on  a  very  specific  culture  where  Marines  deploy.  For  example,  it  pre¬ 
pares  Marines  for  Helmand  Province,  not  Afghanistan. 

CAOCL  has  always  been  the  single  point  of  contact  for  culture  train¬ 
ing  for  the  Marine  Corps’  general  forces.  And,  since  2010,  the  Marine 
Corps  has  mandated  training  for  all  deploying  forces.  Some  of  the 
specific  programs  they  support  include  the  following: 

•  16  hours  on  culture  for  deploying  forces 

•  Additional  modules  available  on  Islam,  gender  roles,  and  the 
drug  economy 

•  Language  programs  in  a  variety  of  different  lengths: 
40/80/120/160  hours,  including  how  to  work  with  interpreters 

•  13  mission-oriented  sets  of  lessons — for  missions  such  as  doing 
a  search  mission,  a  medical  mission,  etc.  (pulled  from  the  Ma¬ 
rine  Corps’  training  manuals) 

•  University  training  provided  by  San  Diego  State  University  and 
University  of  Coastal  Carolina  (8  weeks) 

•  Key  leader  engagement  program — “Leader  +4,”  in  which  a  unit 
commander  and  up  to  four  other  key  individuals  receive  en¬ 
hanced  cultural  training 

CAOCL  has  been  aggressive  in  establishing  an  assessment  approach. 
It  began  in  2006  and  2007  by  surveying  Marines  about  their  language 
and  culture  experiences.  CAOCL  personnel  also  interviewed  many 
returning  Marines  and  looked  at  reports  of  lessons  learned.  They 
used  these  information  sources  to  determine  how  to  develop  the  doc¬ 
trine,  produce  training  manuals,  and  advise  on  strategy. 
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CAOCL’s  assessment  system  has  two  purposes:  (1)  to  find  and  fix 
problems  and  (2)  to  understand  how  the  training  and  materials  ben¬ 
efit  the  Marine  Corps. 

CAOCL  has  found  it  useful  to  develop  an  assessment  platform  that  al¬ 
lows  its  staff  to  use  methods  and  instruments  that  can  provide  differ¬ 
ent  kinds  of  information  for  different  purposes.  They  have  tests  (for 
some  courses)  and  evaluation  forms  at  the  class  level.  They  follow 
what  happens  at  the  practical  exercises,  such  as  Enhanced  Mohave 
Viper  in  Twentynine  Palms,  the  exercise  that  occurs  for  units  just 
about  to  deploy  to  the  Middle  East.  They  have  the  Longitudinal  As¬ 
sessment  Project,  which  uses  qualitative  methods  to  get  insights  from 
a  few  units  before,  during,  and  after  training  events,  during  deploy¬ 
ments  when  possible,  and  after  deployment.  They  also  have  surveys 
that  are  designed  to  gather  Marine  perspectives  on  what  knowledge 
and  skills  are  useful,  how  best  to  teach  these  skills,  and  how  useful 
these  skills  are  for  their  missions. 

CAOCL’s  experience  has  been  that  it  is  not  possible  to  gather  infor¬ 
mation  on  learning/capacity,  especially  if  the  need  is  to  link  it  to  a 
particular  training  event,  using  a  survey  instrument  administered 
days,  weeks,  or  months  after  a  training  event.  There  are  too  many 
confounding  factors,  such  as  learning  from  other  sources,  coupled 
with  a  weak  memory  for  where  and  from  whom  they  learned  some¬ 
thing.  Measuring  learning/ skill  requires  a  carefully  designed  testing 
instrument  and,  potentially,  baseline  instruments.  Even  then,  con¬ 
founding  factors  will  mean  that  links  to  specific  training  events  will  be 
weak.  In  contrast,  surveys  were  useful  for  gathering  information 
about  learning  preferences,  perceptions  of  the  value  and  effective¬ 
ness  of  certain  skills  and  knowledge,  and  the  way  such  skills/ 
knowledge  are  prioritized  in  comparison  with  other  military  training. 
This  can  be  used  to  improve  programs  and  to  make  decisions  about 
application  of  resources  to  different  kinds  of  training. 

After  years  of  discussion  about  the  issue,  CAOCL’s  consensus  is  that 
this  cannot  be  assessed  using  experimental  science  or  quantitative 
measures.  There  are  simply  too  many  confounding  factors,  and  the 
indicators  are  too  variable  across  contexts.  Also,  direct  observation  of 
effectiveness  would  constitute  an  operational  burden  on  Marines. 
While  we  might  be  able  to  come  up  with  something  that  appears  to 
be  quantitative,  it  would  not  be  solid.  Instead,  CAOCL  believes  that  it 


is  more  scientifically  appropriate  to  acknowledge  that  effectiveness 
assessment  is  an  area  where  we  must  rely  on  the  judgment  of  Ma¬ 
rines.  CAOCL  thinks  it  is  unlikely  and  undesirable  to  reach  full  input- 
output  modeling  (e.g.,  battalion  with  training  will  incur  fewer  casual¬ 
ties).  However,  it  is  looking  for  evidence  that  other  types  of  evalua¬ 
tions,  such  as  survey  responses,  attitudes,  or  observations,  can  serve  as 
reasonable  surrogates  for  underlying  performance. 

CAOCL  advised  us  that  any  evaluation  instrument  has  to  be  designed 
with  the  delivery  platform  in  mind.  So,  if  CLREC  plans  to  use  some- 
thing  like  Survey  Monkey-  or  handwritten  responses,  a  simple  design, 
is  essential — one  that  will  limit  what  can  be  assessed.  For  example, 
the  most  basic  options  do  not  have  the  capability  to  do  skip  patterns 
or  reminder  emails.  Likewise,  CLREC  will  need  to  factor  in  the  capa¬ 
bilities  and  time  constraints  of  CLREC  s  staff  to  analyze  results  of  any 
survey  design.  For  example,  if  they  have  time  to  look  only  at  averages 
for  each  question,  the  instrument  needs  to  be  designed  so  that  they 
can  get  some  value  from  that  level  of  analysis. 

Navy  Expeditionary  Combat  Command  (NECC) 

NECC  is  one  of  the  major  users  of  CLREC  language  and  culture 
training  since  many  of  its  units  deploy  overseas.  We  had  discussions 
with  the  language  and  cultural  training  coordinator  on  how  NECC 
develops  its  requirements  and  on  possible  ways  for  obtaining  feed¬ 
back  on  the  suitability  of  the  training  for  its  missions. 

Requirements  are  based  on  Navy  Tactical  Task  (Navy  Mission  Essen¬ 
tial  Task  List)  NTA  4.8.5,  which  requires  military  personnel  to  main¬ 
tain  cultural  awareness.  This  was  the  requirement  that  led  to  the 
founding  of  CLREC,  too.  Most  NECC  personnel  require  OCAT,  but 
some  units  choose  MTT  training  events. 

The  three  organizations  that  require  and  receive  the  most  LREC 
training  are  (1)  Navy  Expeditionary  Intelligence  Command  (NEIC), 
(2)  Maritime  Civil  Affairs  and  Security  Training  Command,  and  (3) 
Construction  Battalions  (Seabees).  The  requirement  for  a  unit  to 


2.  Survey  Monkey  is  a  company  that  provides  web-based  survey  capabilities 
for  people  who  want  to  field  a  survey  themselves.  We  describe  this  alter¬ 
native  for  fielding  CLREC’s  surveys  in  the  final  chapter  of  this  report. 
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have  at  least  one  Dari  speaker  at  the  platoon  level  (McChrystal,  2009 
[20])  created  a  major  hardship  for  NECC.  The  Army  solution — 
computer-based  training — was  not  practical  for  Seabees.  CLREC’s 
staff  member,  Mike  Longoria,  developed  and  delivered  the  required 
curriculum  via  a  live  training  event  to  Seabee  units.  Later,  the  materi- 

3 

al  was  made  available  on  Navy  Knowledge  Online  (NKO) . 

NECC  identified  a  number  of  places  where  assessment  information 
could  be  collected.  CLREC  has  been  doing  after-training  events — 
either  electronically  or  by  paper — following  MTT  events.  NECC  also 
mentioned  that  there  should  be  information  in  commanders’  after¬ 
action  briefs  that  provide  information  on  language  and  cultural  is¬ 
sues.  These  lessons-learned  reports  should  be  available  on  the  classi¬ 
fied  network. 


Army 

We  visited  a  group  of  Army  personnel  in  the  Pentagon  who  work  on 
language  and  culture  training  programs  for  the  Army.  These  Army 
experts  told  us  that  soldiers  who  take  language  and  culture  training 
do  not  know  the  value  of  what  they  have  learned  until  they  experi¬ 
ence  a  deployment  where  soldiers  have  to  apply  their  knowledge. 

The  Army  has  had  difficulty  getting  feedback  on  the  utility  of  its 
courses,  although  feedback  has  improved  now  that  the  Rapport 


3.  However,  CLREC  informed  us  that  it  has  stopped  allowing  downloads  of 
products  on  NKO  because  of  difficulty  obtaining  any  information  on  the 
number  of  downloads,  reasons  for  use,  and  number  of  personnel  using 
the  material.  CLREC  would  probably  make  the  materials  available  again 
if  NKO  could  provide  usage  statistics.  Unfortunately,  there  is  a  moratori¬ 
um  on  new  Navy  websites  (since  2006),  so  CLREC  must  rely  on  other 
Navy  websites  to  distribute  materials  (personal  communication  from 
Pete  Christensen,  June  18,  2012). 

4.  However,  we  looked  at  Marine  and  Navy  lessons-learned  databases  and 
found  that,  although  culture  and  language  training  was  sometimes  men¬ 
tioned,  the  lessons  learned  were  not  very  helpful  for  CLREC  insights  or 
metric  development.  For  example,  one  lesson  was  that  Marines  should 
be  taught  the  dialect  of  the  region  where  Marines  are  going,  rather  than 
the  country’s  official  language. 
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5 

program  is  a  requirement  for  all  soldiers  who  are  going  to  deploy. 
The  Army  has  produced  a  video  that  provides  examples  in  which 
knowledge  of  language  and  culture  was  mission  essential.  In  one  ex¬ 
ample  from  the  video,  an  Army  Medic  describes  how  the  ability  to 
speak  and  understand  Dari  made  a  mission-critical  difference  in  the 
success  of  a  medevac. 

The  Army  language  and  culture  program  is  getting  requests  from 
theater  commanders  to  provide  more  training  to  soldiers.  The  letter 
from  General  McChrystal — specifying  that  each  Army  unit  will  have  a 
soldier  who  is  Level  1  capable  in  Dari  [20] — has  greatly  improved  the 
recognition  of  the  importance  of  language  skills  among  Army  rank 
and  file. 

Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI) 

ARI  has  performed  research  to  support  the  Army’s  cross-cultural 
competence  development.  ARI  analysts  have  reviewed  behavioral 
models  that  can  be  used  to  identify  how  well  people  adapt  to  differ¬ 
ent  cultures  and  what  kinds  of  training  programs  are  most  likely  to 
produce  results  (Abbe,  Gelle,  and  Everett,  2010  [21]).  They  have  re¬ 
viewed  existing  instruments  to  identify  those  that  appeared  most 
promising  for  measuring  cultural  competence  and  have  tested  them 
against  populations  of  enlisted  personnel,  officers,  and  cadets. 

Recently  ARI  staff  have  provided  a  strategic  overview  of  many  of  the 
aspects  of  cross-cultural  competence  as  it  applies  to  the  military,  in¬ 
cluding  its  relationship  to  training  (Caligiuri,  2011  [15]).  They  dis¬ 
cuss  the  importance  of  acquiring  general  skills  for  developing 
competence,  including  time  constraints  and  the  applicability  of 
training.  As  part  of  this,  ARI  has  developed  a  five-stage  model  for  cul¬ 
tural  learning: 


5.  The  Army’s  Rapport  program  is  mandatory  online  basic  language  and 
cultural  awareness  training  in  Dari  and  Pashto  for  soldiers  deploying  to 
Afghanistan.  Iraqi  Rapport  (training  Iraqi  Arabic)  is  mandatory  for  sol¬ 
diers  deploying  to  Iraq.  All  three  languages  were  made  available  in  Oc¬ 
tober  2010,  while  Rapport  is  accessible  to  all  services  via  the  Joint 
Language  University  website  (http: / / www.army.mil/standto/  archive/ 
2010/11/09). 
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•  Identifying — being  aware  of  the  different  cultural  system 

•  Understanding — grasping  principles  of  how  the  different  for¬ 
eign  culture  works 

•  Coping — beginning  to  interact  with  the  other  system  and  be¬ 
ginning  to  learn 

•  Managing — learning  how  to  manage  the  other  system  in  mutu¬ 
ally  acceptable  ways 

•  Integrating — incorporating  selected  elements  into  their  own 
operating  framework 

ARI  also  notes  that  validating  training  effectiveness  needs  new  feed¬ 
back  mechanisms,  such  as  postdeployment  interviews  and  surveys. 


Air  Force 


The  Air  Force  categorizes  personnel  into  one  of  three  tiers  to  deter¬ 
mine  what  level  of  language  and  cultural  training  they  need.  Most 
airmen  are  in  tier  1  and  receive  general  cultural  awareness  training 
since  they  won’t  be  interacting  with  local  nationals.  Tier  2  training  is 
country  specific  and  recently  has  focused  on  either  Iraq  or  Afghani¬ 
stan.  This  training  is  for  airmen  who  will  sometimes  interact  with  lo¬ 
cals.  Tier  3  training  is  for  those  airmen  who  will  interact  often  with 
local  nationals  and  is  delivered  through  human  instruction.  Training 
for  tiers  1  and  2  is  only  available  online. 

Currently  no  one  is  assessing  whether  this  training  is  effective  at  a 
level  higher  than  Kirkpatrick  Level  1,  although  the  Air  Force  Culture 
and  Language  Center  has  developed  a  survey  to  receive  feedback  on 
the  training.  This  center  also  develops  the  curriculum  for  cultural 
training. 

Iraq  and  Afghanistan  have  driven  most  of  the  demand  for  cultural 
training  in  the  Air  Force.  Air  Force  deployments  are  for  six  months. 
Each  airman  usually  has  four  to  five  days  of  administrative  and 
equipment  processes  to  go  through  before  deploying  to  a  foreign 
country.  It  is  possible  that  Africa  and  parts  of  Asia  could  be  useful  to 
understand  in  the  coming  years,  but  there  is  a  sense  that  culture 
training  may  become  less  of  a  concern  after  we  withdraw  from  Iraq 
and  Afghanistan. 
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Summary  and  discussion 

Ovir  review  of  assessments  and  research  related  to  language  and  cul¬ 
ture  training  for  general  forces  identified  several  important  themes. 

First,  every  organization — including  Congress,  GAO,  DOD,  and  the 
services — is  interested  in  assessing,  in  a  systematic  fashion,  the  rela¬ 
tionship  between  language  and  cultural  training  and  mission  effec¬ 
tiveness.  There  have  been  concerns  that  the  existing  systems  have  not 
been  adapting  effectively  to  the  importance  of  LRC  training  that 
SSTRO  actions  require.  Most  of  the  existing  evidence  for  the  need 
for  LRC  training  has  focused  on  anecdotes,  both  good  and  bad,  that 
imply  an  important  relationship  between  training  and  mission  effec¬ 
tiveness — such  as  the  Army’s  example  of  the  medic  whose  medevac 
was  successful  because  of  his  ability  to  understand  Dari. 

Also,  the  need  for  widespread  development  of  LRC  capabilities  will 
remain  after  the  completion  of  OEF.  National  security  policy,  such  as 
outlined  in  the  Quadrennial  Defense  Review,  is  expanding  the  geo¬ 
graphic  areas,  cultures,  languages,  and  capabilities  that  will  be  re¬ 
quired  in  the  future.  The  widespread  desire  from  Congress  and  DOD 
will  be  for  a  more  systematic  approach  for  developing  and  evaluating 
the  needs  for  LRC  training. 

The  assessment  of  training  and  mission  outcomes  requires  at  least 
two  separate  instruments.  Feedback  on  how  well  training  was  deliv¬ 
ered  can  be  performed  immediately  after  the  training  was  completed. 
However,  assessing  whether  the  training  was  effective  in  supporting 
the  mission  objectives  needs  to  be  performed  at  a  later  time.  A  mis¬ 
sion-oriented  assessment  would  ideally  be  done  either  during  de¬ 
ployment  or  immediately  on  completion  of  deployment. 

It  is  not  realistic  to  expect  sailors  (or  other  deployed  personnel)  to 
connect  mission  impacts  related  to  culture,  language,  or  regional  ex¬ 
pertise  directly  to  training.  There  will  be  a  significant  time  gap  between 
training  and  mission  effects.  Also,  individuals  may  have  learned  materi¬ 
al  on  their  own,  from  family  members,  or  while  deployed.  And  they  are 
not  likely  to  be  able  to  connect  specific  training  to  specific  outcomes. 

For  these  and  other  reasons,  it  is  not  likely  that  LREC  training  can  be 
incorporated  into  simple  input-output  models  that  have  been 
developed  to  translate  traditional  military  inputs  into  mission  success 
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outcomes.  The  effects  of  LREC  training  are  too  diffuse  and  difficult 
to  control  for  in  operational  settings. 

Language  metrics  are  more  developed  than  cultural  or  regional  ex¬ 
pertise  metrics.  Also,  measures  for  assessing  low-level  language  exper¬ 
tise  are  being  promulgated  that  will  provide  a  way  to  assess  the  levels 
of  expertise  that  are  appropriate  for  the  general  forces,  as  compared 
with  language  professionals.  However,  the  bulk  of  training  received 
across  all  services  has  been  cultural  and  regional  training. 

The  different  stages  of  a  Kirkpatrick-style  evaluation  are  present  at 
various  places  throughout  DOD.  The  Navy  and  Marine  Corps  evalu¬ 
ate  their  I  .RFC  programs  for  their  learning  content.  And  DOD  has 
collected  some  basic  high-level  assessments  of  mission  benefits.  How¬ 
ever,  no  service  has  yet  produced  sets  of  evaluation  metrics  over  time 
in  an  integrated  fashion. 

Level  2  and  3  evaluations  would  need  to  be  performed  on  a  broad 
Navy  population  to  capture  not  just  who  received  training,  but  specif¬ 
ics  about  what  kinds  of  issues  need  to  be  addressed,  and  how  effective 
current  preparations  have  been.  Such  information  would  need  to  be 
collected  in  some  systematic  fashion,  such  as  an  annual  survey  of 
Navy  personnel. 

It  is  likely  that  self-assessments  will  need  to  be  used  as  the  principal 
indicators  for  assessing  the  results  of  training.  Virtually  all  current  ef¬ 
forts  have  used  feedback  from  personnel  either  while  deployed  or  af¬ 
ter  completing  a  deployment  to  provide  feedback  on  LREC  issues 
and  training. 

A  survey  of  Navy  personnel  with  respect  to  LREC  should  address  at 
least  two  types  of  training — (1)  general  preparation  for  limited  inter¬ 
actions,  such  as  shore  leave,  and  (2)  more  extensive  interactions, 
such  as  working  with  foreign  nationals  or  while  deployed  overseas. 

While  the  Navy  has  provided  training  to  a  lower  percentage  of  per¬ 
sonnel  than  other  services,  Navy  personnel  ranked  highest  among 
the  services  in  satisfaction  with  their  LRC  training  and  above  average 
in  terms  of  beneficial  results. 

We  found  at  least  two  organizations  that  would  be  able  to  perform 
service-wide  surveys  that  could  be  used  to  track  some  Level  2  or  3 
metrics:  DMDC  and  NPRST. 


Chapter  3:  LREC  survey  questions 

In  this  chapter,  we  describe  the  survey  questions  that  we  recommend 
CLREC  ask,  in  a  systematic  fashion,  of  all  Navy  personnel  returning 
from  deployments  (or  of  all  Navy  personnel  who  have  used  CLREC 
products  and  services  returning  from  deployment)  ’  and  an  inter¬ 
view/focus  group  protocol.) 

We  have  developed  variations  on  a  basic  survey  so  that  it  will  be  possi¬ 
ble  to  address  the  major  categories  of  Navy  personnel  who  use 
CLREC  services: 

•  Navy  personnel  returning  from  deployments 

•  Commanding  officers 

•  Flag  officers  who  receive  individual  language  instruction  and 
tutoring 

Some  of  these  Navy  respondents  would  be  returning  from  ground 
deployments  to  the  Middle  East,  whereas  others  would  be  returning 
from  ship  deployments  that  spanned  many  areas  of  responsibility 
(AORs).  In  this  chapter,  we  explain  the  rationale,  strengths,  and 
weaknesses  of  the  recommended  questions.  At  the  end  of  this  chap¬ 
ter,  we  make  further  recommendations  that  will  facilitate  using  the 
survey. 

We  group  our  recommended  survey  questions  in  five  major  sections: 

•  Section  I.  Your  mission  and  interactions  with  foreign  nationals 
during  most  recent  deployment 

•  Section  II.  Challenges/issues  with  culture  and  language  during 
this  deployment 


This  chapter  describes  a  single  survey,  shown  in  appendixes  A  and  B,  in¬ 
tended  for  all  Navy  personnel  returning  from  deployments.  Appendixes 
C,  D,  and  E  provide  surveys  for  smaller  populations  (e.g.,  commanding 
officers  and  flag  officers) . 
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•  Section  III.  Self-assessment  of  culture  and  language  awareness 
and  understanding 

•  Section  IV.  Assessment  of  predeployment  culture  and  language 
training 

•  Section  V.  Demographics  and  operation  supported 

The  remainder  of  this  chapter  describes  what  is  included  in  each 
section. 

Table  2  provides  an  overview  of  the  recommended  survey  questions. 
Some  questions  are  in  red  because  they  are  particularly  important  to 
get  specific  answers  to  assess  the  impact  of  and  needs  for  LREC  train¬ 
ing.  Note  that  our  recommendations  regarding  the  mechanics  of  da¬ 
ta  collection  (where,  when,  how) ,  how  to  report/ aggregate 
responses,  and  how  to  provide  feedback  to  respondents  are  in  chap¬ 
ter  4,  “Alternatives  and  recommendations.” 


Table  2.  Overview  of  recommended  survey  questions 
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Section  I:  Your  mission  and  interactions  with  foreign 
nationals  during  deployment 

The  first  section  of  our  survey  begins  with  questions  about  the  mis¬ 
sion  and  interactions  that  the  respondent  had  with  foreign  nationals. 
The  purpose  is  to  get  the  respondents  thinking  back  to  their  most  re¬ 
cent  deployment,  setting  the  stage  for  other  survey  sections  to  follow. 
Note  that  survey  questions  are  in  boldface  at  the  beginning  of  each 
paragraph.  The  discussion  of  each  question  follows  in  regular  type. 

In  what  AOR  and  country  did  you  most  recently  deploy?  (Drop-down 
menu  of  AORs/ countries)  The  intent  of  the  first  question  is  to  pin¬ 
point  what  areas  of  responsibility  were  included  in  the  respondents’ 
latest  deployment.  This  is  particularly  important  because  it  matches 
the  respondent’s  answers  with  the  Cl  .RFC  AOR  coordinator  who  ar¬ 
ranged  or  provided  any  training  that  the  sailor  might  have  received. 
From  this  first  question,  CLREC  will  be  able  (1)  to  determine  who  at 
CLREC  will  be  most  interested  in  the  response  and  (2)  to  use  the 
survey  responses  to  provide  an  overview  of  CLREC’s  client/ customer 
base. 

What  was  your  primary  mission  on  this  most  recent  deploy¬ 
ment/overseas  assignment?  (Ground  and  Sea  deployment  drop-down 
menus)  This  question  is  meant  to  add  information  about  what  the  re¬ 
spondent  was  actually  doing  during  the  deployment.  The  responses 
from  these  questions  could  be  useful  to  determining  how  well 
CLREC  is  doing  at  supporting  different  kinds  of  primary  missions. 

Did  any  of  your  work  duties  involve  interaction  with  foreign 
nationals?  (Yes/No)  This  query  is  critically  important  for  determin¬ 
ing  what  question  the  respondent  will  be  asked  during  the  rest  of  the 
survey  and  for  reducing  the  burden  on  those  sailors  who  have  not 
had  exposure  to  foreign  nationals  during  their  work  activities.  Re¬ 
spondents  who  answer  “No”  will  be  directed  to  the  next  section  of  the 
survey. 

If  “Yes,”  how  often  did  your  work  duties  involve  interaction  with  for¬ 
eign  nationals?  This  question  is  designed  to  get  an  idea  of  how  large  a 
part  of  the  sailor’s  duties  involved  working  with  foreign  nationals.  It 
will  be  particularly  important  if  the  survey  finds,  for  example,  that 
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there  are  sailors  who  work  daily  with  foreign  nationals  who  are  not 
getting  any  LREC  training  before  deployment  (pinpointing  need) . 

What  types  of  interactions  did  you  have  with  foreign  nationals  while 
performing  your  duties?  We  asked  this  question  to  get  an  idea  about 
how  the  respondent  worked  with  foreign  nationals.  Did  respondents 
use  interpreters?  Did  the  foreign  nationals  speak  English?  Was  the  re¬ 
spondent  responsible  for  working  with  foreign  nationals  alone  or  as 
part  of  a  group? 

Was  your  interaction  with  foreign  nationals  part  of  your  duties,  or 
strictly  part  of  your  liberty  time  while  aboard?  This  question  is  meant 
to  validate  the  earlier  question  about  whether  the  respondents 
worked  with  foreign  nationals,  and  to  get  an  idea  of  how  often  the  re¬ 
spondents  met  with  foreign  nationals,  both  on  liberty  and  as  part  of 
work. 

Section  II  A:  Challenges/issues  with  foreign  nationals'  culture 
during  deployment 

We  developed  this  section  in  response  to  CLREC’s  desire  to  know 
about  what  challenges  their  customers  were  facing  (and  about  which 
CLREC  might  not  be  aware).  For  example,  CLREC  personnel  men¬ 
tioned  to  us  that,  when  they  were  supporting  a  humanitarian  mission 
to  Haiti,  the  culture  training  did  not  mention  that  Haitians  greatly 
value  their  cell  phones — even  more  than  they  value  having  housing. 
This  knowledge  would  have  been  useful  in  planning  and  executing 
this  humanitarian  operation.  For  example,  they  could  have  planned 
to  use  Haitians’  cell  phone  numbers  for  identification  and  communi¬ 
cation  purposes. 

Did  you  experience  any  challenges/problems  or  issues  with  foreign 
nationals’  culture  during  his  deployment?  The  intent  of  this  question 
is  to  determine  whether  there  were  any  problems  that  Cl  .RFC’s  train¬ 
ing  might  address.  Respondents  who  answer  “No”  to  this  question  will 
skip  to  another  part  of  the  survey. 

What  was  the  greatest  cultural  barrier  or  friction  point  that  you  en¬ 
countered  during  your  deployment  and  how  did  you/your  unit  over¬ 
come  the  problem?  Unlike  all  earlier  questions,  this  one  allows  the 
respondent  to  write  in  an  answer  (free  response).  We  included  this 
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question  to  gather  descriptions  of  the  type  of  cultural  problems  that 
sailors  encounter  during  deployment.  When  we  pretested  this  ques¬ 
tion  on  a  humanitarian  assistance  cruise  to  Latin  American  ports 
(“Southern  Partnership  Station”),  we  obtained  an  unanticipated  an¬ 
swer:  More  than  one  respondent  said  that  the  biggest  work-related 
problem  was  the  foreign  military  coming  aboard  ship  unexpectedly 
because  the  Navy  personnel  did  not  know  the  ranks  and  names  of  the 
ranks  in  Spanish.  This  response  demonstrates  that  CLREC  customers 
do  not  know  enough  about  CLREC’s  products,  because  CLREC  pro¬ 
duces  a  product  called  NORTH/SOUTHCOM  Military  Rank  Insignia. 

From  the  standpoint  of  completing  your  mission,  how  significant 
were  the  challenges  and  issues  you  faced  with  foreign  nationals’  cul¬ 
ture  during  this  deployment?  What  was  the  “cost”  of  these  issues,  in 
terms  of  accomplishing  your  mission?  We  wrote  this  question  to  get 
an  idea  of  the  costs  of  not  having  enough  cultural  knowledge  and  the 
benefits  of  having  such  knowledge  (Kirkpatrick  Level  4).  Although 
the  answers  will  not  obtain  monetary  values  for  return-on-investment 
analysis,  it  will  provide  information  that  allows  CLREC  to  do  a  “return 
on  expectation”  analysis. 

From  the  standpoint  of  maintaining  good  relations  with  foreign  civil¬ 
ians,  how  significant  were  the  challenges  and  issues  you  faced  with 
foreign  nationals’  culture?  This  question  is  intended  to  provide  re¬ 
turn-on-expectation  information  about  the  indirect  benefits  of  learn¬ 
ing  about  cultures,  and  the  costs  of  being  without  that  knowledge 
base. 

Section  II  B:  Challenges/issues  with  foreign  nationals' 
language  during  this  deployment 

This  section  makes  the  same  queries  as  does  section  2A,  but  it  asks 
about  language  rather  than  cultural  issues.  We  added  this  section  in 
response  to  CLREC’s  request  that  we  differentiate  problems  with  cul¬ 
ture  from  challenges  with  language.  This  section  starts  with  a  Yes/No 
question  about  whether  there  were  any  language  challenges  faced. 
Respondents  who  answer  “No”  skip  to  a  later  section  in  the  survey. 


34 


Section  III:  Self-assessment  of  individual  knowledge  of  the 
culture  and  language  of  the  foreign  nationals  with  whom  you 
worked 


We  developed  this  section  to  get  a  self-rating  of  the  knowledge  and 
ability  of  respondents  (Kirkpatrick  Level  2).  We  asked  about  six  dif¬ 
ferent  areas:  ability  to  understand  the  language,  ability  to  speak  the 
language,  knowledge  of  cultural  do’s  and  don’ts,  male/ female  proto¬ 
cols,  religious  beliefs,  and  body  language/ gestures.  This  is  followed 
by  a  question  about  the  overall  difficulty  of  communicating  with  for¬ 
eign  nationals  during  this  deployment.  The  final  question  asks  where 
respondents  acquired  their  cultural  and  language  knowledge.  This 
question  was  added  as  a  result  of  our  pilot  testing  on  the  humanitari¬ 
an  cruise.  Several  of  the  respondents  said  that  they  had  high  under¬ 
standing  of  Spanish  but  that  it  was  because  they  grew  up  in 
households  where  Spanish  was  the  primary  language  spoken.  We 
added  this  question  because  we  thought  that  it  was  important  to 
know  the  sources  of  cultural/language  knowledge  and  skills. 

Section  IV:  Assessment  of  your  predeployment  culture  and 
language  training 

This  section  of  the  survey  asks  respondents  to  connect  any  culture/ 
language  training  they  received  with  their  deployment  experiences. 
CAOCL  cautioned  against  using  this  approach  because  they  found 
that  respondents  had  difficulty  remembering  their  predeployment 
training.  We  added  this  section  anyway  because  it  seems  that  the  only 
way  to  honestly  assess  the  impact  of  predeployment  training  is  to  col¬ 
lect  data  after  deployment,  even  if  the  data  are  not  as  believable  be¬ 
cause  of  concerns  about  respondents’  memories.7 

Prior  to  this  deployment,  did  you  receive  any  training  in  cul¬ 
ture/language  of  the  foreign  nationals  with  whom  you  interacted  on 


7.  In  an  ideal  study  design,  we  would  compare  units  that  were  randomly  as¬ 
signed  to  receive  culture  and  language  training  (“experimental  group”) 
with  ones  that  were  randomly  assigned  to  receive  no  such  training 
(“control  group”).  This  design  is  unlikely  to  be  acceptable  to  the  Navy 
for  logistical,  ethical,  and  cost  reasons. 
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this  deployment?  (Yes/No)  This  question  is  designed  to  allow  those 
who  received  no  training  to  skip  to  the  last  section  of  the  survey. 

What  organization  provided  the  culture /language  training  or  training 
products  that  you  received  prior  to  your  deployment?  (Circle  all  that 
apply)  We  added  this  question  to  differentiate  CLREC  training  from 
training  by  other  organizations. 

Please  estimate  the  total  amount8  of  CULTURE  training  that  you  re¬ 
ceived  prior  to  this  latest  deployment  (select  one  response  only) .  We 

asked  this  question  to  get  a  better  understanding  of  how  much  cul¬ 
ture  training  the  respondents  had.  For  example,  it  would  be  more 
understandable  if  earlier  self-ratings  were  low  if  they  had  received 
very  little  training. 

Please  estimate  the  total  amount  of  LANGUAGE  training  that  you  re¬ 
ceived  prior  to  this  latest  deployment  (select  one  response  only) .  This 
is  the  same  question  as  above,  but  substituting  the  word  “Language.” 

We  asked  this  question  to  get  a  better  understanding  of  how  much 
culture  training  the  respondents  had.  For  example,  it  would  be  more 
understandable  if  earlier  self-ratings  were  low  if  they  had  received 
very  little  training. 

Please  mark  the  extent  to  which  the  following  topics  were  covered  in 
the  predeployment  culture /language  training  that  you  received.  We 

ask  about  six  different  areas:  ability  to  understand  and  speak  the  lan¬ 
guage,  knowledge  of  cultural  do’s  and  don’ts,  male/female  protocols, 
religious  beliefs,  and  body  language/gestures.  This  is  the  same  list 
that  we  used  in  the  self-assessments  earlier  in  the  survey.  The  idea  is 
to  determine  how  much  training  respondents  had  in  these  areas. 

Overall,  how  useful  was  the  CULTURE  training  that  you  received? 
(Please  circle  one).  The  purpose  of  this  question  is  to  determine 
whether  the  training  was  actually  used  during  the  deployment  (Kirk¬ 
patrick  Level  3) . 


8.  The  response  alternatives  for  amount  of  training  are  (a)  one  hour  or 
less,  (b)  more  than  an  hour,  up  to  a  day,  (c)  more  than  a  day,  up  to  a 
week,  and  (d)  more  than  a  week  of  training. 


Overall,  how  useful  was  the  LANGUAGE  training  that  you  received? 
(Please  circle  one).  This  question  is  the  same  as  the  one  above,  ex¬ 
cept  that  it  concerns  language,  not  culture.  The  purpose  of  this  ques¬ 
tion  is  to  determine  whether  the  training  was  actually  used  during  the 
deployment  (Kirkpatrick  Level  3) . 

What  did  you  like  best  about  the  training  that  you  received?  (Free  re¬ 
sponse)  We  included  this  free-response  question  to  give  respondents 
a  chance  to  pinpoint  specific  areas  where  their  training  was  particu¬ 
larly  useful  or  enjoyable. 

What  did  you  like  least  about  the  training  that  you  received?  (Free  re¬ 
sponse)  This  free  response  question  was  designed  to  give  respondents 
a  chance  to  pinpoint  particular  areas  where  their  training  was  espe¬ 
cially  useful  or  enjoyable. 

What  improvements  could  be  made  to  help  improve  training?  (Free 
response)  We  included  this  free-response  question  to  give  respond¬ 
ents  an  opportunity  to  suggest  areas  of  improvement  that  did  not 
make  their  list  of  things  they  “liked  best”  or  “liked  least”  about  train¬ 
ing  they  received. 

Section  V:  Demographics  and  operation  supported 

In  this  section  of  the  survey,  we  wanted  to  obtain  more  detailed  in¬ 
formation  about  the  respondents.  This  information  could  be  used  to 
gain  better  understanding  of  who  receives  CLREC  training  and  what 
type  of  deployments  they  have. 

What  is  your  current  service  component?  (Please  circle  one)  CLREC 
should  ask  this  question  to  be  consistent  with  its  Kilpatrick  Level  1 
surveys  and  to  determine  whether  there  is  a  difference  between  the 
responses  of  active  duty  Navy  (USN),  reservists  (USNR),  Coast  Guard, 
and  others. 

9 

If  you  are  currently  in  the  military,  are  you  enlisted  or  officer?  Earli¬ 
er  research  shows  that  officers  and  enlisted  have  different  views  of  the 


9.  We  are  allowing  for  the  possibility  that  some  respondents  will  be  Navy  ci¬ 
vilians  when  they  take  the  survey. 


37 


benefits  of  culture,  regional,  and  language  training.  Therefore,  we 
must  ask  whether  the  respondent  is  officer  or  enlisted. 

If  you  are  enlisted,  please  provide  your  Distributable  Navy  Enlisted 
Classification  (DNEC).  In  this  question,  we  leave  a  space  for  the  re¬ 
spondent’s  first  and  second  DNEC.  We  ask  this  to  determine  whether 
needs  for  CLREC  training  differ  depending  on  the  skill  set  of  the 
sailors  responding. 

If  you  are  an  officer,  please  provide  your  designator  code.  This  ques¬ 
tion  is  meant  to  determine  whether  officers  with  different  designators 
have  different  I  .RFC  training  needs. 

What  is  your  current  rank?  We  found  studies  where  the  ratings  of  the 
usefulness  of  training  were  different  for  senior  military  than  for  jun¬ 
ior  military.  Therefore,  we  ask  about  rank. 

What  is  your  UIC?  This  question  is  asked  because  we  want  to  know 
where  the  sailor  is,  which  is  uniquely  defined  by  Unit  Identification 
Code  (UIC). 

What  is  your  command  and  location?  This  is  a  way  to  identify  the  re¬ 
spondent’s  unit,  for  those  who  do  not  provide  a  proper  UIC  number. 

How  long  did  your  latest  deployment  last?  There  is  some  evidence 
that  longer  deployments  have  different  problems  than  do  shorter 
deployments.  This  is  why  we  want  to  know  deployment  lengths. 

What  is  your  naval  enterprise?  We  ask  this  question  to  find  out 
whether  responses  to  culture  and  language  training  are  different  for 
the  different  naval  enterprises. 

What  is  the  operation/ exercise /deployment  that  you  supported  on 
this  latest  deployment?  We  thought  that  it  would  be  useful  to  know  in 
what  deployment,  exercise,  or  operation  the  respondent  had  just  par¬ 
ticipated.  For  those  who  repeat  on  a  yearly  or  semiyearly  cycle  (e.g., 
Cobra  Gold),  perhaps  a  separate  distinct  curriculum  would  need  to 
be  developed. 

What  unit  did  you  serve  with?  We  asked  this  question  because  we 
know  that  many  of  Cl  .RFC’s  clients  come  from  Seabees,  Military  Civil 
Affairs  Units,  Explosive  Ordnance  Disposal  Units,  Navy  Expeditionary 


Intelligence  Combat  Command,  and  Carrier  Strike  Groups.  It  might 
be  that  some  groups  have  different  culture/language  training  needs 
than  others. 

Summary  and  conclusion 

This  chapter  has  reviewed  the  reasoning  behind  our  recommended 
questions  for  ongoing,  systematic  assessment  of  CLREC  training  and 
training  materials. 

The  data  collection  tool  and  questions  that  we  have  developed 
(appendixes  A  and  B)  obtain  the  basic  information  that  would  be 
most  useful  for  CLREC  to  have  available  when  it  determines  needs 
for  new  or  different  courses,  and  as  it  documents  the  impact  that  it 
has  on  improving  the  job  performance  and  reducing  liberty  incidents 
of  troops  overseas.  These  survey  questions,  however,  could  be  asked 
of  a  number  of  different  customers — not  strictly  of  sailors,  but  also  of 
unit  commanders  and  commanding  officers  (see  appendix  C)  of 
sailors  who  participate  in  culture  and  language  training.  Ideally,  flag 
officers  who  receive  CLREC-coordinated  language  tutoring  should  al¬ 
so  be  surveyed  (see  appendix  E) .  The  next  chapter  summarizes  our 
recommendations  for  which  customers  should  be  surveyed  and  how 
they  should  be  contacted. 
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Chapter  4:  Alternatives  and  recommendations 

In  this  chapter,  we  describe  a  set  of  alternatives  that  CLREC  should 
consider  as  it  takes  the  next  step  in  getting  a  systematic  feedback  on 
the  needs  for,  and  impacts  of,  its  products  and  services. 

Alternatives  for  questions  and  respondents 

Table  3  provides  an  overview  of  the  data  that  CLREC  ideally  needs  in 
order  to  have  a  comprehensive  assessment  capability.  CLREC  might 
decide  that  some  data  collection  alternatives  are  not  needed.  This 
chapter  follows  the  sequence  of  table  3  in  describing  the  alternatives 
for,  and  placing  priorities  on,  different  analyses. 

Survey  of  sailors 

CLREC ’s  largest  numbers  of  customers  are  sailors  who  later  deploy 
around  the  world  as  part  of  sea  or  ground  operations.  The  sailors’ 
units  might  be  participating  in  such  missions  as  Theater  Security  Co¬ 
operation  (TSC),  Humanitarian  Assistance/Disaster  Relief,  Multi- 
National  Force  Operations,  or  a  multitude  of  more  traditional  as¬ 
signments,  such  as  serving  in  a  Carrier  Strike  Group,  working  on  a 
submarine  deployment,  or  assisting  antipiracy  operations.  Some  of 
these  sailors  might  serve  on  the  ground  (e.g.,  a  Judge  Advocate  Gen¬ 
eral  officer  providing  legal  advice  to  a  Marine  Corps  commander  in 
Kandahar)  or  on  the  sea. 

CLREC  is  missing  two  elements  that  it  needs  to  obtain  convincing 
quantitative  information  on  the  impact  of  training  and  needs  for  dif¬ 
ferent  (or  additional  LREC  training):  (a)  an  exhaustive  list  of  indi¬ 
viduals  who  use  CLREC’s  training  and  (b)  surveys  and  interview/ 
focus  groups  to  obtain  details  on  the  challenges  that  their  customers 
have  while  on  deployments.  The  main  questions  CLREC  would  like 
to  ask  follow:  (1)  Did  you  receive  LREC  training?  (2)  If  not,  how 
might  it  have  helped  the  mission?  (3)  If  so,  what  kind  of  CLREC 
training  did  you  receive?  (4)  Did  CLREC  training  help?  (5)  If  not, 
why  not?  and  (6)  How  can  we  improve  CLREC  training? 
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List  of  individuals  requesting  and/or  using  CLREC  services  and 
products  (email  collection  tool) 

CLREC  has  done  an  excellent  job  of  providing  its  products  and  ser¬ 
vices  in  a  wide  variety  of  modalities;  unfortunately,  that  makes  it  more 
difficult  to  obtain  an  exhaustive  list  of  individuals  who  use  their  mate¬ 
rials.  Nevertheless,  CLREC  can  do  the  best  it  can  by  systematically  re¬ 
questing  information  on  customers  from  the  points  of  contact  who 
ask  for  CLREC  courses  and  materials.  Appendix  A  shows  one  format 
for  a  sign-in  sheet  that  would  provide  sufficient  information  for 
CLREC  to  (a)  document  the  number  of,  ranks,  and  units  of  custom¬ 
ers,  and  (b)  provide  email  addresses  that  could  be  used  to  contact 
customers  immediately  after  they  complete  their  deployments.  The 
form  also  asks  sailors  about  how  they  think  they  will  use  the  training 
that  they  have  received. 

In  our  experience  with  other  projects,  sailors  on  vessels  that  are  re¬ 
turning  from  deployments  often  have  some  downtime  when  they 
could  respond  to  a  survey  more  easily  than  they  could  during  an  ear¬ 
lier  part  of  the  cruise.  This  same  format  (shown  in  appendix  A)  could 
be  used  for  sailors  who  are  using  CLREC-supplied  materials,  whether 
the  materials  are  OCAT  DVDs,  courses  on  NKO,  or  supplementary 
materials  provided  to  sailors  serving  as  individual  augmentees  (IAs) 
in  non-Navy  units. 

The  form  in  appendix  A  collects  a  bare-bones  amount  of  information 
on  the  number  and  units  of  individuals  using  Cl  .RFC  materials. 
Therefore,  it  will  help  measure  the  impact  of  CLREC  activities,  but  it 
provides  almost  no  information  on  the  needs  of  sailors  before  de¬ 
ployment — except  for  one  short  question  on  how  the  sailor  intends 
to  use  the  information  he/shejust  received. 

Postdeployment  survey 

CLREC  currently  lacks  a  standard  survey  form  that  will  systematically 
request  postdeployment  information  from  individuals.  Appendix  B 
provides  a  survey  form  that  could  be  used.  This  form,  which  we  ex¬ 
plained  in  chapter  3,  could  obtain  much  more  extensive  information 
on  needs  for  LREC  training  (by  providing  information  on  the  type  of 
missions,  descriptions  of  their  biggest  cultural/language  challenges) , 
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as  well  as  on  the  impact  of  training  (by  gathering  customer  ratings  on 
the  usefulness  of  training  after  deployment  has  been  completed) . 

The  reader  might  wonder  why  we  are  asking  about  the  usefulness  of 
training  after  we  wrote  in  chapter  2  that  customers  are  unlikely  to 
remember  their  training  (or  the  source  of  their  cultural  and  lan¬ 
guage  knowledge)  once  their  deployments  have  ended.  We  answer 
this  two  ways.  First,  even  though  customers  might  not  remember  how 
they  obtained  their  cultural  knowledge,  it  is  important  to  get  a  self- 
assessment  of  skills  to  know  whether  there  are  “holes”  in  the  current 
system  that  Cl  .RFC  might  help  fill.  (In  other  words,  for  the  assess¬ 
ment  of  needs /requirements  for  training,  it  does  not  really  matter 
whether  sailors  adequately  remember  their  training;  however,  the 
self-assessment  will  not  provide  perfect  evidence  for  assessing  the  re¬ 
sults  of  CLREC  training.)  Second,  there  is  currently  no  way  to  know 
what  culture  and  language  deployment  needs  there  are  for  CLREC 
customers  from  the  perspective  of  those  returning  from  deployment. 

The  DMDC  survey  provides  evidence  that  Navy  customers  are  satis¬ 
fied  with  their  LRC  training  (see  chapter  2,  figures  2  through  4),  but 
it  is  not  clear  how  many  of  these  sailors  obtained  their  training  from 
CLREC.  Some  might  have  received  training  from  other  organiza¬ 
tions,  such  as  the  Defense  Language  Institute.  With  benefit  of  a  list  of 
personnel  who  used  CLREC’s  products  (appendix  B),  CLREC  can 
focus  its  efforts  on  getting  feedback  just  from  its  own  customers. 

Survey  of  officers  and  unit  commanders 

Appendix  C  includes  a  survey  form  to  be  sent  to  the  officers  and  unit 
commanders  of  sailors  who  use  CLREC  materials.  Getting  the  email 
addresses  of  officers  and  unit  leaders  would  involve  some  “detective 
work”  on  the  part  of  CLREC  (or  an  organization  CLREC  chooses  to 
perform  the  survey).  The  detective  work  would  involve  using  the 
unit’s  identified  in  the  email  collection  tool  (appendix  B),  and  look¬ 
ing  for  officers  who  lead  those  units.  Using  the  Navy’s  deployment 
databases  and  Navy  global,  CLREC  could  obtain  a  good  list  of  officers 
who  led  sailors  identified  in  the  email  collection  tool. 

The  advantages  of  collecting  responses  to  the  postdeployment  survey 
for  officers  and  unit  commanders  are  that  (1)  officers  and  unit  com¬ 
manders  have  a  wider  view  of  their  sailors’  missions  and  assignments 
than  do  the  seamen  themselves,  (2)  officers  and  unit  commanders 
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know  more  about  the  wider  impact  of  their  sailors’  behavior  than  do 
the  troops  themselves,  and  (3)  officers  and  unit  commanders  know 
about  assignments  that  are  not  made  to  their  sailors,  and  why.  We 
believe  that  without  obtaining  information  from  the  officers  who  lead 
CLREC-trained  troops,  we  would  (a)  neglect  an  important  client  of 
CLREC  services  and  products,  (b)  fail  to  obtain  quotes  from  com¬ 
manders  on  the  benefits  of  knowledge  and  skills  in  culture  and  lan¬ 
guage,  and  (c)  ignore  one  method  by  which  CLREC  could  become 
better  known  by  officers  and  unit  commanders. 

Interviews  of  sailors  and  unit  commanders 

The  fourth  data  collection  tool  that  we  have  provided,  shown  in 
appendix  D,  is  an  interview  protocol  for  both  sailors  and  offic¬ 
ers/unit  commanders  who  lead  sailors  who  have  used  CLREC  ser¬ 
vices  and  products.  We  consider  this  tool  to  be  superior  to  a  survey  in 
some  respects,  and  inferior  to  a  survey  in  others. 

On  one  hand,  a  survey  is  a  good  way  to  obtain  information  (especially 
quantitative  information)  from  a  large  number  of  people  cheaply 
and  efficiently.  It  is  efficient  because  it  can  be  deployed  to  widely  dif¬ 
ferent  physical  locations  by  means  of  an  email  with  a  link  to  the  sur¬ 
vey’s  website.  The  survey  is  efficient  because  it  is  impersonal  and 
automated.  It  is  best  at  providing  quantitative  evidence  of  the  impact 
and  needs  for  CLREC  training. 

On  the  other  hand,  interviews  are  better  than  surveys  at  providing 
details  about  a  topic.  Interviews  allow  the  data  collector  to  ask  follow¬ 
up  questions  if  a  response  is  unclear  or  incomplete — a  survey  does 
not  provide  that  option.  For  example,  let  us  say  that  a  respondent  to 
a  survey  replies  to  “Please  describe  the  changes  that  would  improve 
the  training”  by  stating  that  “we  should  add  more  language  training 
that  would  help  us  deal  with  foreign  military.”  In  a  survey,  you  cannot 
ask  questions  about  what  kind  of  language  training  (understanding, 
speaking,  or  reading),  which  foreign  military,  or  what  particular 
incident  prompted  the  respondent  to  recommend  more  training. 
Does  the  respondent  mean  that  they  need  help  using  interpreters, 


10.  For  example,  an  officer  might  decide  against  giving  an  assignment  to  a 
unit  because  he/ she  believes  that  the  unit  has  insufficient  language  and 
cultural  understanding. 
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with  knowing  the  ranks  of  foreign  military,  or  what?  An  interview, 
however,  provides  an  opportunity  to  probe  for  the  reasons  and  details 
that  a  survey  response  does  not  provide.  In  other  words,  an  interview 
will  provide  better  information  for  getting  qualitative  understanding 
of  training  needs  and  impact. 

Interviews  or  surveys  for  flag  officers 

We  show  our  final  recommended  data  collection  tool  in  appendix  E. 
It  is  designed  to  ask  brief  questions  of  flag  officers  who  had  language 
tutoring  arranged  by  CLREC.  This  set  of  five  questions  is  very  brief 
because  of  flag  officers’  busy  schedules  and  many  demands  on  their 
time. 

We  have  elected  first  to  ask  each  flag  officer  what  training  he  or  she 
received  because  collecting  this  information  will  help  show  what  ser¬ 
vices  the  flag  officer  obtained.  The  second  and  third  questions  ask 
how  much  instructor  time  the  training  used  and  whether  the  flag  of¬ 
ficer  had  been  given  previous  training  in  the  language.  These  two 
questions  were  meant  to  determine  what  level  of  language  training 
the  flag  officer  requested.  The  fourth  question  asks  for  an  evaluation 
of  how  useful  the  language  training  was — a  measure  of  impact.  The 
fifth  and  final  question  asks  for  any  suggestions  on  how  training 
could  be  improved. 

The  protocol  in  appendix  E  would  elicit  richer  data  if  it  could  be 
used  as  an  interview,  but  doing  so  would  require  more  resources  be¬ 
cause  of  the  time  to  schedule  interviews  with  flag  officers  (and  re¬ 
schedule  if  the  flag  officers’  schedules  change),  to  conduct  the 
interview,  and  to  write  and  analyze  the  notes.  It  would  require  far 
fewer  resources  if  it  were  distributed  as  a  survey. 

Alternatives  for  fielding  surveys  or  interviews/focus  groups 

Now  that  we  have  described  what  questions  should  ideally  be  ad¬ 
dressed  to  which  customers  of  CLREC  products,  this  final  subsection 
of  our  report  reviews  various  methods  for  how  these  questions  can  be 
asked,  responses  can  be  collected,  and  analyses  can  be  performed. 
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Fielding  surveys 

Fielding  a  survey  includes  deciding  on  the  sampling  frames,  applying 
for  any  possible  Institutional  Review  Board  (IRB)  considerations, 
contacting  respondents,  collecting  data,  and  analyzing  the  results  of 
those  surveys.  A  decision  about  how  to  field  a  survey  requires  consid¬ 
eration  of  (a)  whether  CLREC  wants  to  have  its  results  (needs,  im¬ 
pact)  compared  with  those  of  other  military  services,  (b)  whether  it 
has  the  staff  capacity  to  field  a  survey  of  returning  deployers  itself,  (c) 
when  it  needs  the  information,  and  (d)  whether  it  wants  to  perform  a 
one-time  survey  or  ongoing  surveys. 

Data  collection  and  sampling  frames 

The  issues  of  data  collection  and  sampling  frames  are  interconnect¬ 
ed.  If  we  assume  that  CLREC’s  first  priority  for  a  survey  is  to  obtain 
feedback  from  sailors  returning  from  deployment,  we  do  not  think 
that  probability  sampling  will  be  a  feasible  option.  Probability  sam¬ 
pling  is  concerned  with  selecting  individuals  from  within  a  popula¬ 
tion  to  estimate  the  characteristics  of  the  entire  population.  It 
requires  a  method  to  define  the  population  and  a  sampling  frame,  or 
list  of  all  target  population  members.  For  example,  a  survey  of  stu¬ 
dents  in  schools  could  have  probability  sampling  with  a  complete  list 
of  the  schools,  and  each  school  having  a  complete  list  of  students. 
This  lends  itself  to  probability  sampling  because  the  list  of  schools 
changes  slowly,  as  does  the  list  of  students.  Furthermore,  students  are 
required  to  attend  school.  The  most  important  advantage  of  proba¬ 
bility  sampling  is  the  ability  to  weight  respondents’  answers  to  esti¬ 
mate  how  the  entire  population  would  have  responded  to  the 
question. 

However,  since  ships  deploy  (and  return  home)  at  difficult-to-predict 
times,  those  times  frequently  change,  and  the  decision  for  a  sailor  to 
participate  is  voluntary,  we  do  not  think  that  probability  sampling  is 
the  best  method  for  CLREC’s  needs:  The  list  of  course  participants 
changes  too  rapidly. 


1 1 .  CLREC  should  not  have  difficulty  with  an  IRB  as  long  as  surveys  confine 
themselves  to  training  issues.  Interviews  and  focus  groups,  however, 
would  require  more  detailed  review  because  of  the  face-to-face  nature  of 
the  interaction  (i.e.,  respondents  are  not  anonymous). 
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Instead,  we  would  recommend  a  survey  to  use  a  convenience  sample 
based  on  one  of  two  methods:  (1)  surveying  all  sailors  who  are  re¬ 
turning  from  deployments  (as  NPRST  has  done)  and/or  (2)  sending 
an  email  to  all  CLREC  course  participants  at  the  email  address. 

Analysis  of  data 

We  favor  surveying  all  sailors  who  are  returning  from  deployment  be¬ 
cause  doing  so  would  allow  a  statistical  comparison  of  responses  of 
CLREC  participants  versus  nonparticipants.  By  using  regression 
techniques,  the  analysis  could  help  describe  (1)  how  returning 
CLREC  participants  are  different  from  returning  non-CLREC  partic¬ 
ipants  and  (2)  what  factors  predict  favorable  responses  on  the  survey 
(saying  that  they  have  a  good  understanding  of  the  cultural  do’s  and 
don’ts,  language,  etc.).  For  example,  analysis  could  explore  whether 
sailors  who  deployed  to  South  America  are  more  favorable  toward 
CLREC  training  than  those  who  deployed  to  another  area  of  the 
world. 

Reporting  results 

CLREC  needs  to  decide  if  survey  results  need  to  be  reported  formally 
(e.g.,  a  technical  report)  or  if  a  less  formal  method  (e.g.,  a  briefing) 
is  desired.  The  more  Cl  .RFC  desires  a  formal  account  of  the  study, 
the  greater  the  need  for  an  outside  organization  to  perform  and  re¬ 
port  results  of  the  survey.  If  a  less  formal  report  is  desired,  it  is  more 
feasible  to  use  CLREC’s  own  staff  resources. 

Feedback  to  participants 

Another  consideration  is  whether  there  could  be  feedback  to  survey 
participants.  It  is  a  nice  touch  to  provide  a  brief  (one-page)  report  to 
survey  participants.  This  habit  of  reporting  back  to  participants  (e.g., 
with  sailors  who  took  CLREC  training)  would  help  spread  awareness  of 
CLREC  and  make  it  more  likely  that  participants  will  respond  to  fu¬ 
ture  surveys  if  they  take  another  CLREC  course. 

Regardless  of  CLREC’s  decisions  regarding  these  questions,  we  rec¬ 
ommend  an  electronically  based  survey  for  returning  deployers  be¬ 
cause  of  the  convenience  of  data  collection  and  the  potential  for 
quick  turnaround  of  survey  findings,  once  the  survey  has  been  ap¬ 
proved  and  set  up,  as  CLREC  requires.  We  think  that  the  portable 
document  format  (pdf)  works  well  for  Level  1  surveys  given  to 


participants  immediately  after  a  course  is  completed,  but  this  elec¬ 
tronic  format  would  be  more  difficult  for  an  instrument  attempting 
to  survey  sailors  returning  from  deployment. 

Cross-service  comparisons 

If  CLREC  wants  to  have  its  data  collected  and  compared  with  other 
organizations  across  services,  DMDC  would  be  the  strongest  choice 
for  fielding  the  survey.  DMDC’s  mission  is  to  provide  and  store  cross¬ 
service  databases,  and  it  has  the  experience  of  fielding  the  Status  of 
Forces  special  survey  on  language,  regional,  and  cultural  training.  It 
is  even  possible  that  DMDC  has  data  from  its  earlier  Status  of  Forces 
survey  that  could  pinpoint  respondents  who  used  CLREC-provided 
services  and  products,  although  we  doubt  that  is  the  case. 

If  CLREC  does  not  want  cross-service  comparisons,  a  natural  alterna¬ 
tive  for  fielding  a  survey  would  be  NPRST,  which  performs  many  sur¬ 
veys  every  year  for  the  Navy  and  its  commands.  One  of  the  advantages 
of  choosing  NPRST  is  that  it  has  its  own  Institutional  Review  Board. 
Furthermore,  it  has  established  particularly  strong  contacts  and  expe¬ 
rience  within  the  Navy  to  conduct  a  Navy-wide  survey,  whether  it  is  a 
special-needs  survey  or  a  matter  of  including  CLREC  questions  on  an 
ongoing  survey  that  NPRST  already  performs. 

Capacity  to  field  survey 

It  is  our  impression  that,  at  present,  CLREC  does  not  have  the  staff 
necessary  to  field  a  survey  on  its  own.  The  organization’s  staff  mem¬ 
bers  are  currently  too  busy  providing  products  and  services  to  spend 
time  collecting  and  analyzing  systematic  survey  data.  Furthermore, 
there  are  advantages  to  having  an  outside  organization  field  and  ana¬ 
lyze  one’s  data,  if  it  is  meant  to  demonstrate  the  impact  of  a  program. 
Outside  organizations  are  considered  more  objective,  evenhanded, 
and  convincing  than  are  program  self-evaluations. 

However,  if  CLREC  chooses  to  field  a  survey  on  its  own,  there  are  off- 
the-shelf  programs,  such  as  Survey  Monkey™,  which  could  make  the 
process  of  fielding  an  electronic  survey  much  easier  than  it  has  been 
in  the  past.  The  cost  for  a  year  of  Survey  Monkey1M  could  be  about 
$300  for  basic  capabilities,  or  $780  with  special  features  and  phone 
support.  However,  it  would  still  require  significant  time  for  staff 
members  to  identify  a  list  of  respondents,  deal  with  bounce-backs  for 


49 


wrong  email  addresses,  write  second-and  third-time  appeals  for  indi¬ 
viduals  to  respond  to  the  survey,  keep  statistical  data  on  response 
rates,  maintain  and  update  the  database  as  responses  come  in,  per¬ 
form  statistical  analyses  on  responses,  perform  qualitative  analyses  on 
free-response  answers,  and  write  and  present  results.  Furthermore,  by 
confining  the  survey  to  participants,  it  would  not  be  possible  to  com¬ 
pare  participants  and  nonparticipants. 

Timeframe  for  needing  survey  results 

The  sooner  that  CLREC  needs  data  to  support  its  impact  and  needs 
for  its  services,  the  more  important  it  will  be  to  use  an  organization 
that  is  well  positioned  to  field  the  survey  and  has  institutional 
knowledge  of  the  respondents  to  be  surveyed.  This  criterion  would 
mean  that  organizations  within  the  Department  of  Navy  would  be 
better  positioned  than  private  contractors  for  this  work.  Using  private 
contractors  would  necessitate  developing  a  request  for  proposal 
(RFP) ,  taking  bids,  and  choosing  among  a  variety  of  organizations  to 
perform  the  work. 

Although  our  discussion  of  organizations  to  field  a  survey  has  pin¬ 
pointed  DMDC  and  NPRST  as  potential  choices,  there  are  a  number 
of  other  organizations  that  could  perform  this  work,  such  as  federally 
funded  research  and  development  centers  (e.g.,  the  Center  for  Naval 
Analyses,  IDA,  and  RAND),  defense  contractors  (e.g.,  SAIC  and  Booz 
Allen  Hamilton),  and  firms  specializing  in  survey  research  (e.g.,  Re¬ 
search  Triangle  Institute,  Westat,  Gallup,  and  Mathematica  Policy 
Research) . 

Fielding  interviews/focus  groups 

It  is  more  expensive  to  collect  data  from  large  numbers  of  people  us¬ 
ing  interviews  or  focus  groups  because  of  the  extensive  hours  re¬ 
quired  to  collect  data  (and  possible  travel  or  phone  bills). 
Nevertheless,  the  data  from  interviews  and  focus  groups  is  likely  to 
produce  more  insights  and  detail  than  could  be  provided  by  a  survey. 
The  organizations  just  mentioned  could  all  perform  interviews  or  fo¬ 
cus  groups,  although  we  would  think  that  with  these  two  data  collec¬ 
tion  methods  it  is  even  more  important  to  use  a  data  collection 
organization  that  has  experience  interfacing  with  military/Navy 
personnel. 
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Summary  and  conclusion 

In  this  research  memorandum,  we  have  reported  the  results  of  our 
interviews  with  organizations  that  have  conducted  previous  assess¬ 
ments  of  language,  regional  expertise,  and  cultural  training,  looking 
for  insights  that  could  be  applied  to  CLREC’s  needs.  Our  analyses  of 
the  interviews  and  the  previous  assessments  yielded  the  following 
conclusions: 

1.  Many  organizations  are  interested  in  assessing  LRC  training, 
but  much  of  the  evidence  reported  (so  far)  involves  anecdotal 
rather  than  systematic  data  collection. 

2.  Assessing  whether  training  was  effective  in  supporting  mission 
objectives  needs  to  be  done  at  a  later  time,  not  immediately  af¬ 
ter  training  has  occurred.  Ideally,  a  mission-oriented  assess¬ 
ment  would  be  performed  during  deployment  or  immediately 
upon  completion  of  deployment. 

3.  It  is  not  realistic  to  expect  sailors  or  other  deployed  personnel 
to  connect  mission  impacts  related  to  culture,  language,  or  re¬ 
gional  expertise  directly  to  training.  It  is  more  realistic  to  ex¬ 
pect  evaluations  of  what  problems  and  challenges  they 
encountered,  how  well  they  met  those  challenges,  and  what 
they  wish  they  knew  before  the  deployment  began. 

4.  Data  and  analyses  will  be  better  at  determining  needs  for  train¬ 
ing  than  at  providing  tightly  defined  input-output  or  return-on- 
investment  models  of  dollars  for  training  and  dollars  of  benefit. 
A  better  goal  would  be  to  provide  return-on-expectation  evi¬ 
dence  showing  that  sailors  were  prepared  and  responded  well 
to  situations  that  required  skills  of  language,  regional  expertise, 
or  cultural  awareness. 

5.  Because  of  the  high  expense  of  developing  tests  of  proficiency, 
it  is  likely  that  self-assessments  will  need  to  be  used  as  a  princi¬ 
pal  indicator  for  assessing  the  results  of  training.  The  profi¬ 
ciency  tests  that  currently  exist  are  narrowly  focused  on 
language  skills  of  specialists  (e.g.,  FAOs,  crypto-linguists,  inter¬ 
preters)  ,  who  form  a  small  part  of  CLREC’s  customer  base. 

6.  While  the  Navy  has  provided  training  to  a  lower  percentage  of 
personnel  than  have  the  other  services,  it  compares  favorably 
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in  terms  of  satisfaction  with  their  LRC  training  and  its  benefi¬ 
cial  results. 

7.  Ideally,  an  assessment  of  Navy  CLREC  training  would  include 
collecting  data  from  sailors  and  their  officers/unit  command¬ 
ers  via  surveys  and  interviews.  Flag  officers  would  be  requested 
to  complete  a  short  five-item  survey  or  interview. 

8.  CLREC  should  prioritize  among  the  five  different  data  collec¬ 
tion  alternatives  provided  in  table  3:  (1)  email  data  collection 
tool,  (2)  postdeployment  survey  for  sailors,  (3)  postdeployment 
survey  for  officers  and  unit  commanders,  (4)  postdeployment 
interviews  with  sailors  and  unit  commanders,  and  (5)  post¬ 
deployment  surveys  of  flag  officers  taking  CLREC-coordinated 
language  tutoring. 

Many  different  organizations  could  field  and  collect  a  survey  for 
CLREC,  including  (but  not  limited  to)  DMDC  and  NPRST,  studies 
and  analysis  federally  funded  research  and  development  centers,  sur¬ 
vey  research  organizations,  and  defense  studies  contractors. 
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Appendix  A:  Email  collection  tool 


The  following  page  shows  an  email  data  collection  tool  for  obtaining 
contact  information  on  course  participants. 
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Sign-in  list  (Illustration) 


Comment: 

How  do  you 
anticipate 
using  this 
training? 

Phone 

e-mail  address  (NIPR  preferred) 

Approximate 
mnth/yr  you 
will  deploy 
(MM  /YY) 

05/12 

Unit 

CLREC 

Rank 

Civ 

Name  (Last,  First) 

Carey,  Neil 
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Q. 

03 

Q- 

03 

> 


> 

> 

03 

C 

c 

CD 

to 

c 

CD 


CD 

Q. 


T3 

T3 

03 

'E 

E 


CD 

i_ 

CUD 

CD 

C 

CJ 

g 

it 

_o 

O 

"o 

4— 

to 

CD 

CD 

O 

_C 

+-» 

~E 

4— 1 

03 

c 

o 

u 

LU 

‘cud 

C£ 
_ 1 

CD 

cr 

CD 

U 

o 

4-» 

O 

u 

c 

03 

T3 

CD 

c 

u 

X 

1— 

c 

E 

i— 

3 

o 

D 

4— 1 

O 

Q 

CD 

to 

c 

O 

CD 

4—* 

_Q 

o 

03 

T3 

u 

E 

Z3 

o 

X 

1 — 

o 

4— 

_£= 

to 

cc 

O 

c 

£ 

X 

o 

4— 

o 

6 

i— 

4— 

4-J 

CD 

to 

4-J 

‘E 

c 

CD 

H 

E 

U 

c 
o 

4-» 
03 
+-> 
LO 

> 
i_ 

o 
u 

cd" 

4-» 
03 

o 

4-> 

u 
CD 

5 

CD 

4-> 
D 

u 

“D 
C 
03 

-  clT 


CD 

Q. 

X 


03 

C 

o 


CD 
D 

c 
CD 

CUD  <5 

CD  ^ 

cc  £ 

CD"  CD 
CUD  _Q 

ro  o 

3  q; 
00  _ 
c  s 

03 
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Pensacola,  FL  32511-5152 


Appendix  B 


Appendix  B:  Postdeployment  survey  for  sailors 

The  next  page  begins  the  Postdeployment  Survey  for  Sailors,  for 
gathering  Level  3-4  information  from  returning  troops. 
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SR  Version 


Language,  Regional  Expertise,  and  Cultural  (LREC) 
Postdeployment/Overseas  Assignment  Survey  for  Sailors, 
Marines,  and  Coast  Guard  Personnel 


Level  3-4  End  of  Deployment  Survey 

This  survey  covers  your  "Cultural  Awareness  Training  Event." 

PRIVACY  ACT  STATEMENT:  Authority  to  request  this  information  is  granted  under  Title 
5,  U.S.  Code  301  and  Department  of  the  Navy. 

PURPOSE:  The  purpose  of  this  questionnaire  is  to  collect  and  compile  data  to  evaluate 
the  effectiveness  of  training. 

ROUTINE  USES:  The  information  collected  will  be  used  by  the  Navy  Foreign  Language 
Office  and  the  Center  for  Language,  Regional  Expertise  and  Culture  (CLREC)  to  evaluate 
training  and  make  improvements. 

DISCLOSURE:  The  information  that  you  provide  WILL  NOT  become  part  of  your 
permanent  record. 

This  survey  will  take  you  approximately  25  minutes  to  complete.  After  clicking  the 
"SUBMIT"  button,  please  wait  until  you  receive  a  response  indicating  that  the  survey  has 
been  submitted.  This  report  will  be  forwarded  electronically  to  CLREC  at 
clrec(S)navy.mil. 
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Language,  Regional  Expertise,  and  Cultural  (LREC) 
Post-deployment/Overseas  Assignment  Survey  for  Sailors, 
Marines,  and  Coast  Guard  Personnel 


We  are  looking  for  feedback  on  your  most  recent  deployment  or  overseas  assignment,  on  behalf  of  the 
Navy's  Center  for  Language,  Regional  Expertise  and  Culture  (CLREC).  The  purpose  of  this  survey  is  to 
learn  more  about  the  types  of  missions  that  might  require  culture  and  language  training,  and  what 
training  might  be  required.  Your  feedback  will  be  used  to  improve  the  usefulness  of  CLREC's  products 
and  services. 


I.  Your  mission  and  interaction  with  foreign  nationals  during  your  deployment/ 
overseas  assignment _ 


1.  To  what  AOR  and  country  did  you  most  recently  deploy? 

A.  USCENTCOM  (Select  the  one  in  which  you  spent  the  MOST  time  ON  YOUR  LATEST 
DEPLOYMENT): 


O  Afghanistan 

©  Jordan 

©  Pakistan 

O  Bahrain 

©  Kazakhstan 

©  Qatar 

O  Egypt 

©  Kyrgyzstan 

©  Saudi  Arabia 

O  Iran 

©  Kuwait 

©  Syria  Yemen 

O  Iraq 

©  Lebanon 

O  Israel 

©  Oman 

.  USPACCOM  (Select  the  one  in  which  you  spent  the  MOST  time  ON  YOUR  LATEST 
DEPLOYMENT): 

©  Australia 

©Laos 

©  Samoa 

O  Bangladesh 

©Malaysia 

O  Singapore 

O  Bhutan 

©Maldives 

O  Solomon  Islands 

O  Brunei 

©Marshall  Islands 

©  South  Korea 

©  Burma/Myanmar 

©Micronesia 

©  Sri  Lanka 

O  Cambodia 

©Mongolia 

©  Thailand 

Q  China 

©Nauru 

©  Timor-Leste 

©Fiji 

©Nepal 

©Tonga 

O  India 

©New  Zealand 

©Tuvalu 

©  Indonesia 

©North  Korea 

©  Vanuatu 

©Japan 

©Palau 

©  Vietnam 

©Jordan 

©Papua  New  Guinea 

©  Kiribati 

©Philippines 
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C.  USSOUTHCOM  (Select  the  one  in  which  you  spent  the  MOST  time  ON  YOUR  LATEST 
DEPLOYMENT): 


Q  Argentina 

O  Cuba/Guantanamo 

0  Honduras 

O  Belice 

Bay 

O  Nicaragua 

0  Bolivia 

O  Dominican  Republic 

O  Panama 

0  Brazil 

0  Ecuador 

0  Paraguay 

0  Chile 

0  El  Salvador 

0  Peru 

0  Columbia 

0  Guatemala 

0  Suriname 

O  Costa  Rica 

O  Guyana 

O  Uruguay 

i.  USAFRICOM  (Select  the  one  in  which  you  spent  the  MOST 
DEPLOYMENT): 

time  ON  YOUR  LATEST 

O  Algeria 

O  Gabon 

O  Nigeria 

O  Angola 

O  Gambia 

0  Rwanda 

O  Benin 

O  Ghana 

O  Sao  Tome  and  Principe 

O  Botswana 

O  Guinea 

O  Senegal 

O  Burkina 

O  Guinea-Bissau 

0  Seychelles 

O  Burundi 

O  Ivory  Coast 

0  Sierra  Leone 

O  Cameroon 

O  Kenya 

0  Somalia 

O  Cape  Verde 

O  Lesotho 

0  South  Africa 

O  Central  African 

O  Liberia 

O  South  Sudan 

Republic 

O  Libya 

O  Sudan 

O  Chad 

O  Madagascar 

O  Swaziland 

©  Comoros 

O  Malawi 

O  Tanzania 

O  Congo 

O  Mali 

O  Togo 

O  Democratic  Republic 

O  Mauritania 

©Tunisia 

of  Congo 

O  Mauritius 

O  Uganda 

O  Djibouti 

O  Morocco 

O  Zambia 

O  Equatorial  Guinea 

0  Mozambique 

O  Zimbabwe 

0  Eritrea 

O  Namibia 

O  Ethiopia 

O  Niger 
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E.  USEUCOM  (Select  the  one  in  which  you  spent  the  MOST  time  deployed  IN  YOUR  LATEST 
DEPLOYMENT): 


o  Albania 
O  Andorra 
o  Armenia 
o  Austria 
O  Azerbaijan 
o  Belarus 
o  Belgium 
O  Bosnia  and 
Herzegovina 
O  Bulgaria 
o  Croatia 
c  Cyprus 
O  Czech  Republic 
O  Denmark 
o  Estonia 
©  Finland 


O  France 
O  Georgia 
O  Germany 
O  Greece 
O  Hungary 
O  Iceland 
O  Ireland 
O  Italy 
O  Latvia 
O  Liechtenstein 
O  Lithuania 
Q  Luxembourg 
O  Macedonia 
O  Malta 
O  Moldova 
O  Monaco 


O  Montenegro 
O  Netherlands 
O  Norway 
Q  Poland 
O  Portugal 
Q  Romania 
O  San  Marino 
O  Serbia 
0  Slovakia 
0  Slovenia 
0  Spain 
O  Sweden 
O  Switzerland 
0  Ukraine 
O  United  Kingdom 
O  Vatican  City 


F.  USNORTHCOM  (Select  the  one  in  which  you  spent  the  MOST  time  deployed  IN  YOUR 
LATEST  DEPLOYMENT): 


O  Antigua  and 

O  Dominican 

o 

Nicaragua 

Barbuda 

Republic 

o 

Panama 

O  Bahamas 

O  El  Salvador 

o 

Saint  Kitts  and  Nevis 

©  Barbados 

O  Grenada 

o 

Saint  Lucia 

O  Belize 

O  Guatemala 

o 

Saint  Vincent  and  the 

0  Canada 

O  Haiti 

Grenadines 

O  Costa  Rica 

O  Honduras 

o 

Trinidad  and  Tobago 

O  Cuba 

0  Jamaica 

O  Dominica 

O  Mexico 

2.  What  was  your  primary  mission  on  this  most  recent  deployment/overseas  assignment? 
(Please  select  one): 


(a)  On  the  ground  (drop  down  List  ONE) 

O  Direct  military  action/patrols 
O  Staff  headquarters 
0  Humanitarian  assistance/disaster  relief 
O  Theater  security  cooperation 
O  Multinational  Force  ops 
O  Other  (specify): _ 


(b)  At  sea  (drop  down  List  TWO) 

O  Direct  military  action/maneuvers 
O  Staff  work  at  sea 

O  Humanitarian  assistance/disaster  relief 
O  Theater  security  cooperation 
O  Multinational  Force  ops 
O  Other  (specify): _ 
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3. 


Did  ANY  of  your  work  duties  involve  interaction  with  foreign  nationals?  (Select  one) 


O  Yes  O  No  (If  "No,"  please  skip  to  question  #7) 

4.  If  "Yes,"  how  often  did  your  work  duties  involve  interaction  with  foreign  nationals? 

O  Less  than  once  a  month  O  Once  a  week 

O  Once  a  month  O  Daily 

o  Several  times  a  month 

5.  What  types  of  interactions  did  you  have  with  foreign  nationals  while  performing  your 
duties?  (Please  select  "Yes"  to  all  that  apply,  if  any): 

A.  Did  you  use  an  interpreter  while  performing  your  duties? 

O  Yes  O  No 

B.  Did  the  foreign  nationals  speak  English  to  you? 

O  Yes  O  No 

C.  Did  you  interact  with  foreign  nationals  as  part  of  a  group  of  Americans  and  our  allies 
while  performing  your  duties? 

O  Yes  O  No 

D.  Did  you  have  individual  interaction  with  foreign  nationals,  where  you  did  (a)  not  have  an 

interpreter  and  (b)  were  not  part  of  a  group? 

C  Yes  O  |\|o 

6.  Did  you  also  interact  with  foreign  nationals  while  on  liberty  (for  example,  port  calls)? 

O  Yes  O  No 


.  A.  Challenges/issues  with  foreign  nationals’  CULTURE  during  this  deployment 


7.  Did  you  experience  any  challenges/problems  or  issues  with  foreign  nationals'  culture  during 
this  deployment?  (Please  select  one): 

O  Yes  O  No  (If  "No,"  please  skip  to  question  #  13) 

8.  If  "Yes",  were  these  challenges/problems  with  foreign  nationals'  culture  concerning: 


Uniformed  foreign  nationals 
Foreign  civilians 


O  yes 
o  Yes 


O  No 

O  No 


9.  When  did  you  experience  challenges/problems  with  foreign  nationals'  culture  during  this 
deployment? 


As  part  of  my  work  responsibilities 

O  Yes 

O  No 

During  my  time  off,  away  from  work 

O  Yes 

O  No 

During  both  my  work  and  my  time  off 

OYes 

O  No 

10.  What  was  the  greatest  cultural  barrier  or  friction  point  you  encountered  during  your 
deployment  and  how  did  you/your  unit  overcome  the  problem? 


11.  From  the  standpoint  of  completing  your  mission,  how  significant  were  the  challenges  and 
issues  you  faced  with  foreign  nationals'  culture  during  this  deployment? 

O  Not  at  all  significant  O  Very  significant 

O  Not  very  significant  O  Extremely  significant 

O  Somewhat  significant 

12.  From  the  standpoint  of  maintaining  good  relations  with  foreign  civilians,  how  significant 
were  the  challenges  and  issues  you  faced  with  foreign  nationals'  culture? 

O  Not  at  all  significant  Q  Very  significant 

O  Not  very  significant  Q  Extremely  significant 

Q  Somewhat  significant 


II.  B.  Challenges/issues  with  foreign  nationals’  LANGUAGE  during  this 
deployment 


13.  Did  you  experience  any  challenges/problems  or  issues  with  foreign  nationals'  LANGUAGE 
during  this  deployment?  (Please  select  one): 

O  Yes  O  No  (If  "No,"  please  skip  to  question  #19) 

14.  If  "Yes,"  were  these  challenges/problems  with  foreign  nationals'  language  concerning: 

Uniformed  foreign  nationals  O  Yes  O  No 

Foreign  civilians  Q  Yes  O  No 

15.  When  did  you  experience  challenges/problems  with  foreign  nationals'  culture  during  this 
deployment? 


As  part  of  my  work  responsibilities 

O  Yes 

QNo 

During  my  time  off,  away  from  work 

OYes 

O  No 

During  both  my  work  and  my  time  off 

O  Yes 

O  No 
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16.  What  was  the  greatest  language  barrier  or  friction  point  you  encountered  during  your 
deployment  and  how  did  you/your  unit  overcome  the  problem? 


17.  From  the  standpoint  of  completing  your  mission,  how  significant  were  the  challenges  and 
issues  you  faced  with  foreign  nationals'  language  during  this  deployment? 

O  Not  at  all  significant  O  Very  significant 

O  Not  very  significant  Q  Extremely  significant 

o  Somewhat  significant 

18.  From  the  standpoint  of  maintaining  good  relations  with  foreign  civilians,  how  significant 
were  the  challenges  and  issues  you  faced  with  foreign  nationals'  language? 

O  Not  at  all  significant  O  Very  significant 

O  Not  very  significant  O  Extremely  significant 

O  Somewhat  significant 


III.  Self-assessment  of  your  individual  knowledge  of  the  CULTURE  and 
LANGUAGE  of  the  foreign  nationals  where  you  were  deployed 


19.  Please  rate  your  cultural  and  language  awareness  and  understanding  using  the  scale  below. 


No 

awareness 

A  little 

awareness 

Some 

awareness 

A  lot  of 

awareness 

Almost  complete 

awareness 

Ability  to  understand  spoken  language 

o 

o 

o 

o 

o 

Ability  to  speak  the  language 

o 

o 

o 

o 

o 

Knowledge  of  cultural  do's  and  don'ts 

o 

o 

o 

o 

o 

Male/female  protocols 

o 

o 

o 

o 

o 

Religious  beliefs 

o 

o 

o 

o 

o 

Body  language/gestures 

o 

o 

o 

o 

o 

Other  (specify): 

o 

o 

o 

o 

o 
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20.  If  you  have  significant  awareness  of  the  culture  and  language  where  you  deployed,  where 
did  you  acquire  this  knowledge?  (Please  check  all  that  apply) 


□ 

□ 

□ 

□ 

□ 

□ 


I  learned  about  this  culture  and  language  as  a  child  growing  up. 

I  had  pre-deployment  training  in  culture  and  language. 

I  had  served  previous  deployments  in  this  area  of  the  world. 

I  taught  myself. 

Other  (please  specify): _ 

Does  not  apply— I  do  not  have  significant  awareness  of  the  culture  and  language  where  I 
deployed. 


21.  Overall,  how  difficult  was  it  for  you  to  communicate  with  foreign  nationals  during  this 
deployment? 


IV.  Assessment  of  your  pre-deployment  culture/language  training  for  interacting 
with  foreign  nationals* 


(*Note—This  section  is  only  for  those  who  received  pre-deployment  culture/language  training.  If  you 
did  not  have  such  pre-deployment  training,  please  skip  to  question  #32.) 

22.  Prior  to  this  deployment,  did  you  receive  training  in  the  culture  or  language  of  the  foreign 
nationals  with  whom  you  interacted  during  this  mission?  O  Yes  o  No 

If  "Yes,"  what  kind  of  training  did  you  receive? 


If  "No,"  please  skip  to  question  #32. 

23.  What  organization  provided  the  culture/  language  training  or  training  products  that  you 
received  prior  to  your  deployment?  (Select  all  that  apply): 

Center  for  Language,  Regional  Expertise  and  Culture  (CLREC) 

Defense  Language  Institute  (DLI) 

Naval  Cryptologic  Services  (NCS) 

Partner  Language  Training  Center  Europe  (PLTCE) 

Other  government  provider 
Civilian  contracting  vendor 

Other  (specify)  : _ 

I  do  not  know  what  organization  provided  my  culture  and  language  training  or  training 
products 


□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 
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24.  Please  estimate  the  TOTAL  amount  of  CULTURE  training  that  you  received  prior  to  this  latest 
deployment.  (Select  one  response  only): 

O  One  hour  or  less  of  training 

O  More  than  one  hour  of  training,  up  to  one  day  of  training 
O  More  than  one  day  of  training 
O  More  than  one  week  of  training 

25.  Please  estimate  the  TOTAL  amount  of  LANGUAGE  training  that  you  received  prior  to  this 
latest  deployment.  (Select  one  response  only): 

O  One  hour  or  less  of  training 

O  More  than  one  hour  of  training,  up  to  one  day  of  training 
O  More  than  one  day  of  training 
O  More  than  one  week  of  training 

26.  Please  mark  the  extent  to  which  the  following  topics  were  covered  in  the  pre-deployment 
culture/language  training  that  you  received: 


No  training 

One  hour  or 

less  of 
training 

More  than  one 

hour  of 
training,  up  to 
one  day 

More  than 
one  day  of 
training 

More  than 

one  week  of 
training 

Ability  to  understand  spoken  language 

o 

© 

o 

o 

o 

Ability  to  speak  the  language 

o 

o 

o 

o 

o 

Knowledge  of  cultural  do's  and  don'ts 

o 

o 

o 

© 

o 

Male/female  protocols 

o 

o 

© 

o 

o 

Religious  beliefs 

o 

© 

o 

o 

o 

Body  language/gestures 

o 

© 

o 

o 

o 

Other  (specify): 

o 

o 

o 

© 

o 

27.  Overall,  how  useful  was  the  CULTURE  training  that  you  received  (did  you  use  it  while 
deployed)?  (Please  select  one): 

O  Very  useful  O  Not  at  all  useful 

O  Somewhat  useful  O  Not  applicable 

O  Not  very  useful 
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28.  Overall,  how  useful  was  the  LANGUAGE  training  that  you  received  (did  you  use  it  while 
deployed)?  (Please  select  one): 

QVery  useful 
O  Somewhat  useful 
O  Not  very  useful 

29.  What  did  you  like  best  about  the  culture  and/or  language  training  that  you  received? 


O  Not  at  all  useful 
O  Not  applicable 


30.  What  did  you  like  the  least  about  the  culture  and/or  language  training  that  you  received? 


31.  What  improvements  could  be  made  to  the  culture  and/or  language  training  that  you 
received? 


V.  Your  demographics/unit 


32.  What  kind  of  unit  did  you  deploy  with? 

OSeabees  O  Military  Civil  Affairs  Unit 

QNavy  Expeditionary  Intelligence  Q  Explosive  Ordnance  Disposal  Unit 

OCommand  QCarrier  Strike  Group 

OOther  (Specify): _ 

33.  What  is  your  current  service  component?  (Please  select  one): 

OUSN  O  Coast  Guard 

O  USNR  OOther 
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34.  If  you  are  currently  in  the  military,  which  best  describes  you?  (Please  circle  one): 

O  Enlisted  Q  Contractor 

0  Officer  Q  Government  Employee 

35.  If  you  are  enlisted,  please  provide  your  DNECs: 

a.  DNEC1: _ 

b.  DNEC2: _ 

36.  If  you  are  an  officer,  please  provide  your  designator  code: _ 

37.  What  is  your  current  rank?  (Please  select  one) 

O  E1-E2  O  01-02 

O  E3-E4  O  03-04 

O  E5-E6  O  05-06 

O  E7  or  higher  O  07  or  higher 

O  Other  (specify): _ 

38.  What  is  your  current  UIC? _ 

39.  What  is  your  current  command  and  location?  _ 

40.  How  long  was  your  last  deployment?  (Please  select  one): 

0  1  week  or  less 

o  Over  1  week,  but  no  more  than  3  months 
O  Over  3  months,  but  less  than  6  months 
o  6  months  or  more 

41.  What  is  your  Naval  Enterprise? 

O  Surface  O  Aviation 

O  Subsurface  O  Expeditionary 

0  Other  (Specify): _ 

42.  What  is  the  Operation/Exercise/Deployment  you  supported? 

O0EF 
O  OIF 

O  Ulchi  Freedom  Guardian 
O  Other  (Specify): _ 


O  Balikatan 
O  Cobra  Gold 


O  CW01 
O  CW02 
O  CW03 
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SUBMIT 


THANK  YOU  FOR  PROVIDING  FEEDBACK  ON  YOUR  DEPLOYMENT  AND  CHALLENGES 
REQUIRING  KNOWLEDGE  OF  CULTURE  AND  LANGUAGE. 
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Appendix  C 


Appendix  C:  Postdeployment  survey  for 
officers  and  unit  commanders 


The  following  page  starts  the  Postdeployment  Survey  for  Officers  and 
Unit  Commanders,  which  collects  Level  3-4  information  from  lead¬ 
ers  of  returning  sailors. 
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SR  Version 


Language,  Regional  Expertise,  and  Cultural  (LREC) 
Postdeployment  Survey  for  Officers  and  Unit  Commanders 


We  are  looking  for  feedback  on  your  most  recent  deployment,  on  behalf  of  the  Navy's  Center  for 
Language,  Regional  Expertise  and  Culture  (CLREC).  We  want  to  know  more  about  the  types  of  missions 
that  might  require  culture  and  language  training.  We  are  especially  interested  in  learning  about  your 
insights,  as  an  officer  or  unit  commander,  into  the  needs  for  culture  and/or  language  training  during 
your  current  (or  most  recent)  deployment.  This  feedback  will  be  used  to  improve  the  usefulness  of 
CLREC's  products  and  services. 


i.  Your  mission  and  interaction  with  foreign  nationals  during  your  deployment 


1.  In  what  country  (or  countries)  did  you  most  recently  deploy? 


2.  What  kind  of  primary  mission  did  your  unit  perform  during  this  deployment? 

O  Serving  as  part  of  a  crew  aboard  a  ship 

O  Serving  as  part  of  a  land-based  command  that  included  work  with  foreign  military  personnel 
O  Serving  as  part  of  a  land-based  command  that  included  work  with  foreign  civilians 
O  Other  (please  specify): _ 

3.  During  this  deployment,  how  often  did  personnel  under  your  command  interact  with  the 
following  types  of  foreign  nationals  as  part  of  their  mission?  (Please  make  a  selection  for  each 
row) 


Never 

Less  than  once 

a  week 

Once  a  week 

More  than  once 

a  week 

Every  day 

Foreign  military  personnel 

o 

o 

o 

o 

o 

Foreign  civilian  government 
personnel 

o 

o 

o 

o 

o 

Foreign  civilians 

o 

o 

o 

o 

o 

Other  foreign  personnel 

o 

o 

o 

o 

o 
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4.  During  this  deployment,  how  often  did  personnel  under  your  command  interact  with  the  following 
types  of  foreign  nationals  during  their  time  off  duty  ?(Please  make  a  selection  for  each  row) 


Never 

Less  than  once 

a  week 

Once  a  week 

More  than  once 

a  week 

Every  day 

Foreign  military  personnel 

o 

o 

o 

o 

o 

Foreign  civilian  government 
personnel 

o 

o 

o 

o 

o 

Foreign  civilians 

o 

o 

o 

o 

o 

Other  foreign  personnel 

o 

o 

o 

o 

o 

5.  What  was  the  most  common  primary  language  that  these  foreign  nationals  spoke? 


6.  What  other  languages  did  these  foreign  nationals  speak,  if  any? 


7.  What  types  of  interactions  did  you,  as  a  unit  commander,  have  with  foreign  nationals  while 
performing  this  mission?  (Please  select  all  that  apply): 

A.  Did  you  use  an  interpreter  while  performing  this  mission?  Q  Yes  O  No 

B.  Did  you  interact  with  foreign  nationals  as  part  of  a  group  of  Americans  and  our  allies  while 

performing  this  mission?  O  Yes  O  No 

C.  Did  you  have  individual  interaction  with  foreign  nationals,  where  you  did  not  have  an 

interpreter  and  were  not  part  of  a  group?  O  Yes  O  No 

D.  Overall,  how  often  did  you  interact  with  foreign  nationals  during  this  mission? 

(Please  select  one): 

O  Never 

o  Less  than  once  a  month 
O  Once  a  month 
O  Several  times  a  month 
O  Once  a  week 
O  Daily 
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II.  Challenges/issues  with  foreign  nationals’  culture  or  language  during  this 
deployment 


8.  Did  you  (or  your  unit)  experience  any  challenges  or  issues  with  foreign  nationals'  culture  or 

language  during  this  deployment?  (Select  one)  0  Yes  Q  No 

9.  If  "Yes,"  please  describe  the  most  important  challenges  or  issues  you  or  your  unit  faced  with 
culture  or  language  during  this  deployment.  (This  can  be  for  interaction  as  part  of  the  immediate 
mission,  or  during  off-duty  hours.) 


10.  From  the  standpoint  of  completing  your  mission,  how  significant  were  the  cultural  and  language 
challenges  and  issues  you  or  your  unit  faced?  (Please  select  one): 

Not  at  all  significant  Very  significant 

Not  very  significant  Extremely  significant 

Somewhat  significant 


11.  From  the  standpoint  of  maintaining  good  relations  with  foreign  civilians,  how  significant  were  the 
challenges  and  issues  you  or  your  unit  faced  with  foreign  nationals'  culture  or  language?  (Please 
select  one): 


o 

o 

o 


Not  at  all  significant 
Not  very  significant 
Somewhat  significant 


o 

o 


Very  significant 
Extremely  significant 
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III.  Assessment  of  your  unit’s  knowledge  of  the  culture  and  language  of  the 
foreign  nationals 


12.  Please  rate  your  unit's  cultural  awareness  and  understanding  of  the  foreign  nationals  using  the 
scale  below.  (Please  make  a  selection  for  each  row): 


No 

awareness 

A  little 

awareness 

Some 

awareness 

A  lot  of 

awareness 

Almost  complete 

awareness 

Ability  to  understand  spoken 
language 

o 

o 

o 

o 

o 

Ability  to  speak  the  language 

o 

o 

o 

o 

o 

Knowledge  of  cultural  do's  and 
don'ts 

o 

o 

o 

o 

o 

Male/female  protocols 

o 

o 

o 

o 

o 

Religious  beliefs 

o 

o 

o 

o 

o 

Body  language/gestures 

o 

o 

o 

o 

o 

Other  (specify): 

o 

o 

o 

o 

o 

13.  If  your  unit  had  significant  awareness  of  the  culture  and  language  where  you  deployed,  how  did 
members  of  the  unit  acquire  this  knowledge?  (Please  select  all  that  apply): 

O  They  learned  about  this  culture  and  language  as  children  growing  up. 

O  They  had  pre-deployment  training  in  culture  and  language. 

O  They  had  served  previous  deployments  in  this  area  of  the  world. 

O  They  taught  themselves. 

0  Other  (please  specify): _ 

O  Does  not  apply— my  unit  did  not  have  significant  awareness  of  the  culture  and  language. 

14.  Overall,  how  difficult  was  it  for  members  of  your  unit  to  communicate  with  foreign  nationals 
during  this  deployment? 

O  Extremely  difficult  O  Very  easy 

O  Very  difficult  0  Extremely  easy 

O  Somewhat  easy 
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IV.  Assessment  of  your  unit’s  pre-deployment  culture/language  training 


(*Note— This  section  is  only  for  those  whose  units  received  pre-deployment  culture/language 
training.  If  your  unit  did  not  have  such  pre-deployment  training,  please  skip  to  question  #21.) 


15.  Prior  to  this  deployment,  did  your  unit  receive  training  in  the  culture  or  language  of  the  foreign 
nationals  with  whom  you  interacted  during  this  mission?  O  Yes  O  No 

If  "Yes,"  what  kind  of  training  did  your  unit  receive? 


If  "No,"  please  skip  to  question  #30. 


16.  Did  your  unit  receive  any  culture  training,  language  training,  or  both  prior  to  this  deployment? 

Culture  training  only 

Language  training  only 

Both  culture  and  language  training 

I  do  not  know  what  kind  of  culture  or  language  training  they  had,  if  any 


17.  What  organization  provided  the  culture  and  language  training  or  training  products  that  your  unit 
received  prior  to  your  deployment? 


o 

o 

o 

o 

o 

o 

o 


Center  for  Language,  Regional  Expertise  and  Culture  (CLREC) 

Defense  Language  Institute  (DLI) 

Naval  Cryptologic  Services  (NCS) 

Partner  Language  Training  Center  Europe  (PLTCE) 

Civilian  contracting  vendor 

Other  (please  specify): _ 

I  do  not  know  what  organization  provided  the  culture  and  language  training  or  training 
products 


18.  Please  estimate  the  TOTAL  amount  of  culture  training  that  your  unit  received  prior  to  this 
deployment.  (Select  one  response  only) 


o 

o 

o 

o 


One  hour  or  less  of  training 

More  than  one  hour  of  training,  up  to  one  day  of  training 
More  than  one  day  of  training,  up  to  one  week 
More  than  one  week  of  training 
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19.  Please  estimate  the  TOTAL  amount  of  language  training  that  you  received  prior  to  this 
deployment.  (Select  one  response  only) 

O  One  hour  or  less  of  training 

O  More  than  one  hour  of  training,  up  to  one  day  of  training 
o  More  than  one  day  of  training,  up  to  one  week 
O  More  than  one  week  of  training 

20.  Please  mark  the  extent  to  which  the  following  topics  were  covered  in  the  pre-deployment 
culture/language  training  that  your  unit  received.  (Please  make  a  selection  for  each  row): 


No 

A  little 

Some 

A  lot  of 

Almost  complete 

awareness 

awareness 

awareness 

awareness 

awareness 

Ability  to  understand  spoken 
language 

o 

o 

o 

o 

o 

Ability  to  speak  the  language 

o 

o 

o 

o 

o 

Knowledge  of  cultural  do's  and 
don'ts 

o 

o 

o 

o 

o 

Male/female  protocols 

o 

o 

o 

o 

o 

Religious  beliefs 

o 

o 

o 

o 

o 

Body  language/gestures 

o 

o 

o 

o 

o 

Other  (specify): 

o 

o 

o 

o 

o 

21.  Overall,  how  useful  was  the  CULTURE  training  that  your  unit  received?  (Select  one) 

Very  useful  Not  very  useful 

Somewhat  useful  Not  at  all  useful 

22.  Overall,  how  useful  was  the  LANGUAGE  training  that  your  unit  received?  (Select  one) 

O  Very  useful  O  Not  very  useful 

O  Somewhat  useful  0  Not  at  all  useful 

23.  Do  you  have  any  suggestions  for  how  the  training  your  unit  received  could  be  made  more  helpful? 

Yes 

No 

Not  sure 
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24.  If  you  have  any  suggestions  for  how  your  unit's  cultural  and  language  training  could  be  improved, 
please  describe  them  here. 


25.  Did  your  unit  receive  any  pre-deployment  training  in  culture  or  language  from  the  Navy's  CLREC? 
(If  "No,"  please  skip  to  question  #30) 

o  Yes 

O  No 

o  Don't  Know 

26.  If  your  unit  received  training  from  the  Navy's  CLREC,  was  it  culture  training,  language  training,  or 
both? 

O  Culture  training  only 
O  Language  training  only 
O  Both  culture  and  language  training 
o  I  do  not  know 

27.  How  useful  was  any  CULTURE  training  that  your  unit  received  from  CLREC? 

O  Very  useful  O  Not  at  all  useful 

o  Somewhat  useful  O  Not  applicable 

O  Not  very  useful 

28.  How  useful  was  any  LANGUAGE  training  that  your  unit  received  from  CLREC? 

O  Very  useful  O  Not  at  all  useful 

O  Somewhat  useful  O  Not  applicable 

o  Not  very  useful 

29.  Do  you  have  any  suggestions  for  improving  CLREC  training?  If  so,  please  describe  them  here. 
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V.  Your  demographics/unit 


30.  What  is  your  current  service  component?  (Please  select  one) 

USN 

USNR 

Coast  Guard 
Other 

31.  If  you  are  currently  in  the  military,  which  best  describes  you?  (Please  select  one) 

Enlisted 

Officer 

Civilian 

32.  If  you  are  enlisted,  please  provide  your  DNECs: 

a.  DNEC1: _ 

b.  DNEC2: _ 

33.  If  you  are  an  officer,  please  provide  your  designator  code: _ 

34.  What  is  your  UIC? _ 

35.  What  is  your  command  and  location? _ 

36.  During  this  deployment,  was  your  unit  primarily  active  duty,  reservists,  or  civilians?  (Please  select 
one) 

Active  duty 

Reservists 

Civilians 

37.  How  long  did  this  deployment  last? 
o  1  week  or  less 

o  Over  a  week,  but  less  than  3  months 
O  Over  3  months,  but  less  than  6  months 
o  6  months  or  more 


THANK  YOU  FOR  PROVIDING  FEEDBACK  ON  YOUR  DEPLOYMENT  AND  CHALLENGES  REQUIRING 

KNOWLEDGE  OF  CULTURE  AND  LANGUAGE 

If  you  have  questions  or  concerns  about  this  questionnaire,  please  refer  them  to  Neil  Carey,  Ph.D.,  at 
703-824-2356  or  careyn(5)cna.org. 
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Appendix  D 


Appendix  D:  Postdeployment  interviews  for 
sailors  and  unit  commanders 


The  next  page  starts  the  interview  protocol  for  collecting  post¬ 
deployment  data  from  sailors  and  their  unit  commanders. 
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SR  Version 


Language,  Regional  Expertise,  and  Cultural  (LREC) 
Postdeployment  Survey  for  Sailors  and  Unit  Commanders 


We  are  looking  for  feedback  on  your  most  recent  deployment,  on  behalf  of  the  Navy's  Center  for 
Language,  Regional  Expertise  and  Culture  (CLREC).  The  purpose  of  this  survey  is  to  learn  more  about  the 
types  of  missions  that  might  require  culture  and  language  training.  Your  feedback  will  be  used  to 
improve  the  usefulness  of  CLREC's  products  and  services. 


I.  Your  mission  and  interaction  with  foreign  nationals  during  your  deployment 


1.  What  kind  of  mission  did  you  perform? 


2.  Did  your  deployment  involve  interaction  with  foreign  nationals?  O  Yes  Q  No 

(If  "No,"  please  skip  to  question  #15) 

3.  If  "Yes,"  was  your  interaction  with  foreign  nationals  part  of  the  mission,  or  was  it  strictly  part  of 
being  abroad  during  "free  time"  (for  example,  during  port  calls  or  shopping  trips)? 


4.  What  country  or  region  did  these  foreign  nationals  come  from? 


5.  What  types  of  interactions  did  you  have  with  foreign  nationals  while  performing  this  mission? 

A.  Did  you  use  an  interpreter  while  performing  this  mission?  Q  Yes  O  No 

If  "Yes,"  how  did  you  use  an  interpreter  while  performing  this  mission? 


B.  Did  you  interact  with  foreign  nationals  as  part  of  a  group  of  Americans  and  our  allies  while 
performing  this  mission? 

O  Yes  O  No 

If  "Yes,"  how  did  you  interact  as  part  of  a  group  while  performing  this  mission? 
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C.  Did  you  have  individual  interaction  with  foreign  nationals,  where  you  did  not  have  an 
interpreter  and  were  not  part  of  a  group?  O  Yes  O  No 

If  "Yes,"  please  describe  the  individual  interactions  you  had  with  foreign  nationals. 


D.  How  often  did  you  interact  with  foreign  nationals  during  this  mission? 


II.  Challenges/issues  with  foreign  nationals’  culture  or  language  during  this 
deployment 


6.  Did  you  experience  any  challenges/problems  or  issues  with  foreign  nationals'  culture  or  language 

during  this  deployment?  O  Yes  O  No 

(If  "No,"  please  skip  to  question  #12) 

7.  If  "Yes,"  please  describe  the  most  important  challenges  or  issues  you  faced  with  culture  or 
language  during  this  deployment.  (This  can  be  during  interactions  as  part  of  the  immediate 
mission,  or  during  non-mission  "free  time.") 


8.  Were  the  most  important  challenges  you  faced  ones  of  culture,  language,  or  both?  (Select  one) 

o  The  most  important  challenges  we  faced  involved  culture  only. 

O  The  most  important  challenges  we  faced  involved  language  only. 

O  The  most  important  challenges  we  faced  involved  both  culture  and  language. 

O  Other  (please  specify): _ 

9.  How  important  were  the  challenges  and  issues  you  faced  with  foreign  nationals'  culture  or 
language? 


10.  Did  these  challenges  interfere  with  your  ability  to  accomplish  your  mission?  Yes 


No 
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11.  If  "Yes,"  please  describe  how  culture  and/or  language  challenges  interfered  with  your  ability  to 
accomplish  your  mission. 


III.  Self-assessment  of  your  knowledge  of  the  culture  and  language  of  the  foreign 
nationals  with  whom  you  worked. 


12.  How  much  did  you  know  about  the  foreign  nationals'  culture  and  language  at  the  beginning  of 
this  deployment? 


13.  Please  rate  your  cultural  awareness  and  language  understanding  in  the  following  areas.  (Please 
make  a  selection  for  each  row.) 


No 

awareness 

A  little 

awareness 

Some 

awareness 

A  lot  of 

awareness 

Almost  complete 

awareness 

Ability  to  understand  spoken 
language 

o 

o 

o 

o 

o 

Ability  to  speak  the  language 

o 

o 

o 

o 

o 

Knowledge  of  cultural  do's  and 
don'ts 

o 

o 

o 

o 

o 

Male/female  protocols 

o 

o 

o 

o 

o 

Religious  beliefs 

o 

o 

o 

o 

o 

Body  language/gestures 

o 

o 

o 

o 

o 

Other  (specify): 

o 

o 

o 

o 

o 

14.  Overall,  how  difficult  was  it  for  you  to  communicate  with  foreign  nationals  during  this 
deployment? 
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IV.  Assessment  of  your  pre-deployment  training  for  interacting  with  foreign 
nationals 


15.  Prior  to  this  deployment,  did  you  receive  any  training  in  the  culture  or  language  of  the  foreign 
nationals  with  whom  you  interacted  (or  might  have  interacted  with)  during  this  mission? 

O  Yes  O  No 

(If  “No",  please  skip  to  question  #26) 

If  "Yes,"  what  kind  of  training  did  you  receive? 


16.  Did  you  have  culture  training,  language  training,  or  both? 

Culture  training  only. 

Language  training  only. 

Both  culture  and  language. 

Other  (please  specify): _ 

17.  What  organization  provided  the  culture  and  language  training  or  training  products  that  you 
received  prior  to  your  deployment? 


18.  Please  estimate  the  TOTAL  amount  of  culture  training  that  you  received  prior  to  this  deployment. 
(Please  select  only  one  response) 

O  One  hour  or  less  of  training 

o  More  than  one  hour  of  training,  up  to  one  day  of  training 
O  More  than  one  day  of  training,  up  to  one  week  of  training 
O  More  than  one  week  of  training 

19.  Please  estimate  the  TOTAL  amount  of  language  training  that  you  received  prior  to  this 
deployment.  (Please  select  only  one  response) 

O  One  hour  or  less  of  training 

O  More  than  one  hour  of  training,  up  to  one  day  of  training 
o  More  than  one  day  of  training,  up  to  one  week  of  training 
O  More  than  one  week  of  training 
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20.  Please  mark  the  extent  to  which  the  following  topics  were  covered  in  the  pre-deployment 
culture/language  training  that  you  received.  (Please  make  a  selection  for  each  row.) 


No  training 

One  hour  or 
less  of  training 

More  than 

one  hour  of 
training,  up 
to  one  day 

More  than 
one  day  of 
training 

More  than  one 
week  of  training 

Ability  to  understand  spoken 
language 

o 

o 

o 

o 

o 

Ability  to  speak  the  language 

o 

o 

o 

o 

o 

Knowledge  of  cultural  do's  and 
don'ts 

o 

o 

o 

o 

o 

Male/female  protocols 

o 

o 

o 

o 

o 

Religious  beliefs 

o 

o 

o 

o 

o 

Body  language/gestures 

o 

o 

o 

o 

o 

Other  (specify): 

o 

o 

o 

o 

o 

21.  Overall,  how  useful  was  the  CULTURE  training  that  you  received? 

Very  useful  (Why?) _ 

Somewhat  useful  (Why?) _ 

Not  very  useful  (Why?) _ 

Not  at  all  useful  (Why?) _ 

Not  applicable  (I  received  no  culture  training) 

22.  Overall,  how  useful  was  the  LANGUAGE  training  that  you  received? 

O  Very  useful  (Why?) _ 

O  Somewhat  useful  (Why?) _ 

o  Not  very  useful  (Why?) _ 

O  Not  at  all  useful  (Why?) _ 

O  Not  applicable  (I  received  no  language  training) 

23.  Do  you  have  any  suggestions  for  how  the  training  you  received  could  be  made  more  helpful? 

Yes 

No 

Not  sure 
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24.  If  you  have  any  suggestions  for  how  your  training  could  be  improved,  please  describe  them  here: 


25.  Did  you  receive  any  pre-deployment  training  in  culture  or  language  from  the  Navy's  Center  for 
Language,  Regional  Expertise  and  Culture  (CLREC)?  (Select  one) 

O  Yes 
O  No 

o  Don't  Know 

26.  If  you  received  training  from  the  Navy's  CLREC,  how  useful  was  the  training  that  you  received? 

O  Very  useful  (Why?) _ 

O  Somewhat  useful  (Why?) _ 

o  Not  very  useful  (Why?) _ 

O  Not  at  all  useful  (Why?) _ 


V.  Your  demographics/unit 


26.  What  is  your  current  service  component?  (Please  select  one) 

O  USN 
O  USNR 
O  Coast  Guard 
O  Other 

27.  If  you  are  currently  in  the  military,  which  best  describes  you?  (Please  select  one) 

0  Enlisted 
O  Officer 
Q  Civilian 

28.  If  you  are  enlisted,  please  provide  your  DNECs: 

a. DNECl: _ 

b. DNEC2: _ 

29.  If  you  are  an  officer,  please  provide  your  designator  code: _ 

30.  What  is  your  UIC? _ 

31.  What  is  your  command  and  location? _ 
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32.  During  this  deployment,  was  your  unit  primarily  active  duty,  reservists,  or  civilians?  (Please  select 
one) 

o  Active  duty 
o  Reservists 
O  Civilians 

33.  How  long  did  this  deployment  last? 

O  1  week  or  less 

o  Over  a  week,  but  less  than  3  months 
o  Over  3  months,  but  less  than  6  months 
o  6  months  or  more 


THANK  YOU  FOR  PROVIDING  FEEDBACK  ON  YOUR  DEPLOYMENT  AND  CHALLENGES  REQUIRING 

KNOWLEDGE  OF  CULTURE  AND  LANGUAGE. 

If  you  have  questions  or  concerns  about  this  questionnaire,  please  refer  them  to  Neil  Carey,  Ph.D.,  at 
703-824-2356  or  careyn(5>cna.org. 
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Appendix  E 


Appendix  E:  Postdeployment  survey  for  flag 
officers 


The  following  page  displays  a  one-page  svirvey  for  flag  officers  who 
have  received  CLREC-coordinated  language  tutoring. 
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Language,  Regional  Expertise,  and  Cultural  (LREC) 
Postdeployment  Survey  for  Flag  Officers 


The  purpose  of  this  questionnaire  is  to  give  feedback  to  the  Navy's  Center  for  Language,  Regional 
Expertise  and  Culture  (CLREC)  on  the  usefulness  of  the  language  training  it  coordinates  and  provides. 
This  feedback  will  be  used  to  improve  the  usefulness  of  CLREC's  products  and  services. 

1.  What  kind  of  language  training  did  you  receive  that  was  arranged  by  CLREC?  (Please  specify  the 
language(s)  for  which  you  received  training.) 


2.  How  much  total  instructor  time  did  you  receive  for  your  language  training?  (Please  select  one): 

O  No  instructor  time 
0  An  hour  or  less  of  instructor  time 
0  More  than  an  hour,  up  to  8  hours  of  instructor  time 
O  More  than  8  hours  of  instructor  time,  up  to  one  week 
O  More  than  one  week  of  instructor  time 

3.  Did  you  have  any  other  training  in  the  language  before  the  CLREC  training?  (Please  select  all  that 
apply): 

o  I  already  had  some  background  in  the  language. 

O  I  had  previous  in-theater  training  in  the  language, 
o  I  had  previously  taught  myself  the  language. 

O  No,  I  had  not  had  any  previous  training  in  this  language. 

4.  How  useful  was  the  CLREC  language  training  that  you  received?  (Please  select  one): 

O  Very  useful  O  Not  very  useful 

O  Somewhat  useful  O  Not  at  all  useful 

O  Don't  know— (for  example,  I  have  not  had  a  chance  to  use  my  language  training) 


5.  If  you  have  any  suggestions  for  how  your  training  could  be  improved,  please  describe  them  here: 
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